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Unintrusive Communication of Status in a Packet 
Network in Heavy Traffic 


By G. J. FOSCHINI* 
(Manuscript received October 31, 1983) 


We consider a packet switched network in the situation where communi- 
cation resources are used close to capacity. Such heavy traffic may seem to 
present a dilemma. On one hand, at each node, the usefulness of status 
information about queues at other nodes is manifest. On the other hand, since 
the limitation of transmission resources causes backup, heavy load seems to 
be the worst situation in which to expend still more communication resources 
to convey status information. Under extremely general assumptions on inter- 
arrivals and services, a scaling appropriate for queueing processes in networks 
under heavy traffic has been established. Under these assumptions, we dem- 
onstrate that the status of the entire network can be communicated throughout 
the network, perfectly, in real time, without influencing the scaled queueing 
process. So, within a precise mathematical setting, we see that there is no 
dilemma: status can be conveyed at a negligible cost in a network operating 
at heavy load. Most of today’s computer networks are designed for light-to- 
moderate loading. Yet heavy traffic analysis is growing in relevance, as is 
explained. A brief introduction to the subject of convergence of queueing 
systems is included. 


Il. INTRODUCTION AND RESULT 
1.1 Status information and the contention for communication resources 


The key concern in the mathematical theory of computer commu- 
nication networks is the stochastic contention for limited network 
resources and the associated queueing and delay processes. (See Refs. 
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1 through 5 for basic expositions.) Queueing theory has provided the 
approach to the subject of stochastic resource contention. The waiting 
lines in a computer network are represented as a vector Q(t) = (Q,(t), 
Q2(t), --- , Qx(t)), where Q,(t) represents the number of items queued 
for the kth resource. The dynamics of the network are associated with 
the law of evolution of Q(t). 

In this paper, we focus on the contention for communication re- 
sources. We assume that the network is a packet switched network 
where transmission is ‘the bottleneck resource, and we consider the 
operation of the network in the condition of heavy traffic. By heavy 
traffic we mean that the demand for communication resources is 
approaching the network’s capability for communication, so that 
queues are very large. 

Status information refers to the information needed to describe 
queueing (or delay) as it evolves in time at each node. At moderate 
traffic levels, the issue of communication of status appears formidable. 
(Some initial investigations are reported in Ref. 6.) How much infor- 
mation about the various components of Q(t) should be communicated 
to other nodes? Is there a point of diminishing returns beyond which 
the communication channels become choked with status information, 
thereby significantly worsening the queueing problem? Questions such 
as these are very difficult to approach with queueing theory methods. 
However, under the assumption of heavy traffic, we show that the 
status issue crystallizes and becomes mathematically tractable. 

The circumstance of heavy traffic emphasizes what seems to be the 
dilemma associated with the communication of nodal status informa- 
tion. On the one hand, during times of heavy traffic, such status 
information is especially useful. If at each node in the network there 
is information available about all the queues at every other node, this 
information could be used to control the flow of packets within the 
network and to judiciously control access to the network. On the other 
hand, the limitation of transmission resources causes queueing in the 
first place; therefore, during heavy loading, it may seem inadvisable 
to expend still more communication resources to convey status infor- 
mation. 

We show that there is no dilemma. In the context of a precise 
mathematical model, the status information of the entire network can 
be transmitted throughout the network without any substantial taxing 
of communication resources. Specifically, one can send enough infor- 
mation to describe the queueing situation as it develops in real time. 
Yet the utilization of communication resources for status transmission 
as compared to the utilization of communication resources for trans- 
mitting other packets tends to zero as the traffic increases. 

Figure 1 illustrates the main result in its simplest form. Two 
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Fig. 1—Simple version of main result. 


queueing systems collocated at B represent packets waiting for trans- 
mission resources. General independent sequences of positive random 
variables are assumed to govern interarrival and transmission times. 
Both systems are assumed to be in heavy traffic. In a sense that we 
will make exact, the following holds: Packets can be added to thuse on 
system 2 such that the recipient of the packets at location A can 
reconstruct q(t) perfectly in real time. The process qo(t) remains the 
same after the status packets are added. 


1.2 Heavy traffic scaling 


Showing that status can be conveyed unintrusively involves analyz- 
ing the scaling appropriate for the description of queues (or delays) 
associated with the condition of heavy load. As will be explained in 
the following sections, it has been established that in heavy traffic a 
continuous path process q(t) replaces the jump process Q(t) as the 
natural descriptor of the number of items queued. Roughly, the nor- 
malization involved in obtaining q(t) is a sort of mathematical “lens” 
used to place the essential features of the Q(t) process in focus as the 
traffic is increased. If one did not normalize, the queueing process 
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would tend to infinity in the heavy traffic limit. Furthermore, if one 
does not normalize in just the right way, the resulting process trivial- 
izes to be identically infinity or identically zero. 

One reason why the communication of the queueing situation is 
easier in the heavy traffic limit is that it is not necessary to convey 
Q(t) exactly. The limiting expression of status, q(t), is insensitive to 
the fine structure of the approximating processes, so that it is not 
necessary to communicate each arrival and departure and the exact 
time of occurrence in order to communicate status. Yet the process 
q(t) possesses sample paths of such elaborate structure that the result 
that negligible transmission resources are needed to communicate 
status in the limit may be surprising, or even counterintuitive. 


1.3 Content of sequel 


In the following we discuss the growing relevance of heavy traffic 
models. The paper is written so as to make the result accessible to 
readers unfamiliar with the specialized subject of convergence of 
queueing systems in heavy traffic and, at the same time, provide a 
concise introduction to this topic. This introductory material, pre- 
sented in Sections II and III, gives the basis for a detailed probabilistic 
analysis in Section IV. This analysis develops the main result that the 
various effects associated with the heavy traffic limit can be resolved 
to allow a communication scheme providing unintrusive transmission 
of status. The literature providing the derivation of the foundation 
material is cited for readers who would like more detail. 

For readers already familiar with the heavy traffic theory of packet 
networks, we mention at this point a key feature of our main result in 
Section IV. Namely, from the status information received at a node 
about a queue at any other network node, an approximation to the 
queueing process sample path can be constructed so that in the heavy 
traffic limit the distance between the queueing process sample path 
and its approximation converges in probability to zero. Moreover, the 
transmission of the approximation can be accomplished unintrusively 
in the sense that the limiting queueing process is unchanged in 
distribution by the additional packets transmitted to convey the ap- 
proximations. 

In Section V, with the status communication issue obviated, we 
broach the next level of inquiry dealing with use of status information 
to reduce queueing and delay. 


Il. THE HEAVY TRAFFIC MODEL AND ITS RELEVANCE 


In the heavy traffic model for the network queueing process, the 
components of q(t) = (qi(t), --- , gx(t)) represent the time evolution 
of normalized queue sizes in the limit as the rate A at which work 
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enters the network approaches U, the ultimate rate at which the 
network can do work. If U — A = A/Vn, where A is a positive constant, 
and if Q(t) is the aforementioned K vector of queue sizes, then q(t) = 
lim, .»Q(nt)/Vn. The process q(t) is an example of a diffusion process. 
A precise definition of the limit is discussed in Section III. 

A mathematically equivalent way to obtain the same limit is to 
compose Q(t)/Vn and replace U and A by nU and nA, respectively. 
The first procedure, involving Q(nt)/Vn, relates to a situation where, 
in a given network, operation close to capacity is taking place. The 
Q(t)/Vn situation relates to a sequence of networks, each accommo- 
dating a new demand, nA. (There is nothing special about n. Any 
function of n that increases without bound would yield the same limit.) 
Such a sequence is of interest if the demand for data communications 
grows substantially over a period of many years. The transmission 
capabilities will grow to keep pace with the demand. If processing 
power and memory are inexpensive relative to transmission cost, 
transmission will be the bottleneck. The behavior of the limit of Q(t)/ 
Vn will be descriptive of long-term tendency of the network queueing 
processes. 

Network design issues that have no crisp resolution when addressed 
in the context of today’s networks under nominal operating conditions 
can have a clear answer in the context of the heavy traffic limit. The 
issue of the value of transmitting status information is an example, as 
we shall see. The result we will prove is a very general one in that the 
network can be quite arbitrary, as for example Reiman and Harrison’s 
generalization of the Jackson-type network,’ where each exogenous 
interarrival process, as well as each service process, is modeled as an 
independent, identically distributed (i.i.d.) sequence with a general 
nonnegative distribution. 


2.1 Motivation for diffusion models in computer network theory 


Mathematically, diffusions are finite dimensional vectors whose 
components are continuous random time functions with the property | 
that at each point in time the statistical law concerning future evolu- 
tion depends only on the present value of the function. (Diffusion 
evolution can also depend on time, but we shall not use this degree of 
flexibility. References 8 and 9 are basic references on diffusions.) 
Diffusion representation of behavior is a natural representation to 
consider in instances where the randomness stems from a large number 
of cumulative influences. Diffusion models are used to replace alter- 
native models, which are useful in addressing small-scale phenomena 
but are too cumbersome for analyzing large-scale behavior. In com- 
puter networks the demand for network resources establishes the scale 
of the queueing processes. The queues can be large enough to require 
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a new scale that no longer tracks each quantum jump (i.e., arrival or 
departure). 


2.2 Weiner process with drift 


In this subsection we mention for future reference some diffusions 
of special interest in the theory of computer networks, namely the 
reflecting Weiner processes with drift. Although one can point out 
significant usefulness of other diffusions,’° the following are the dif- 
fusions that are relevant in much of computer network theory. 

Let A and o be constants where o > 0. By the Wiener process 
W(A, o*) with drift A (and dispersion o”) we mean the continuous 
path Gaussian process starting at w(0) (a constant) with E[w(t,) 
— w(t)] = A(t — t) and E[(w(ti) — Ew(t))(w(t2) — Ew(t2))] = 
o’min(t;, te). This Wiener process, as it stands, is often unsuited to 
addressing heavy traffic queueing problems. After all, a queue cannot 
be negative. The Wiener process can be modified to keep the sample 
paths positive. This brings us to the subject of reflection. For A = 0, 
the reflecting Wiener process W(0, a”) (reflected about the zero state) 
is defined to be w(t) = | w(t)|. For A # 0, | w(t)| is not appropriate 
for describing reflection since | w(t) | has drifts of opposite sign on the 
sets {t| w(t) > 0} and {t| w(t) < 0}. For the reflecting Wiener process, 
we want a continuous path process whose incremental behavior 
matches that of w(t) but with the constraint of nonnegativity. The 
variant w(t) = w(t) — min[0, mini, w(t)] is the natural definition for 
the reflecting Wiener process. As long as w(t) > 0, then w(t) has the 
same differential properties as w(t). The subtractive term serves to 
keep the process nonnegative. The notation W(A, a’) is used to denote 
the reflecting Wiener process. 

Vector versions of W and W have been defined where A is:a vector 
and o” is a positive definite matrix. In the vector case, to complete the 
description, angles of reflection must be specified’ at the boundaries 
that prevent the process components from being negative. 


2.3 Importance of diffusion models 


In the context of today’s computer networks, the condition of heavy 
traffic represents an extreme condition. We discuss reasons why 
consideration of this extreme is worthwhile. 

First, the maturation of demand for computer network services is 
not in sight. The accelerating demand coupled with the rapid techno- 
logical advances associated with network components is likely to foster 
a rapid evolution of networks accommodating more and more demand. 
As demand for network resources grows, economies of scale will 
encourage the operation of networks at higher utilizations. This is 
most easily seen in the context of a single M/M/1 queue.” Recall the 
mean delay formula D = (u — )~!. Fix the mean delay D, and it is 
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apparent that as the demand, X, increases, » increases by a smaller 
amount, so that p = (A/n) — 1. (See Ref. 12 for a related discussion.) 

From the delay formula, we see that if \ increases with slope bounded 
away from zero, the increase in p relative to \ that is needed to make 
D negligible goes to zero. Note, however, from the formula for mean 
queue size Q = (1 — p)~!, that the memory resources needed to queue 
packets do not enjoy this economy of scale. In packet communication 
systems where D is small, one should consider whether packet voice 
services should be accommodated. If the answer is yes, \ grows at a 
still faster rate. In such circumstances, one would anticipate a sequence 
of queueing processes behaving more like the Wiener process ideali- 
zation. 

Another reason diffusion models are relevant is that for a specific 
network operating at moderate utilization it is important to under- 
stand behavior in crisis situations. Emergencies can arise, for example, 
if a node crashes and the surviving network attempts to accommodate 
the resulting overload. Another cause can be a community of customers 
that suddenly present the network with an unanticipated level of 
demand. Diffusion methods for tracking the degradation of service 
and providing the capability of inquiring into which method of oper- 
ation yields the most graceful degradation are especially useful. 

Certain constructions give rise to processes that tend to be charac- 
terized by a W process. While today the quiescent operation of packet 
networks is not the condition of heavy load, we have cited above 
influences that motivate a long-term importance for W processes in 
the theory of computer networks. Sometimes, however, because it is 
extraordinarily difficult to analyze certain computer networks, a more 
precise model is replaced by an associated diffusion simply to gain 
tractability. Such heuristics are usually accompanied by measurements 
or simulations,’°!* or else the diffusion can be a very useful “carica- 
ture”"* of the real situation. Looking at the diffusion counterpart of a 
situation arising in practice in a context of light to moderate load, one 
can get an answer that, when interpreted in the original loading, gives 
the correct result.’” Sometimes diffusion provides special insight’® so 
that a new result in the realm of light to moderate traffic is first 
discovered by obtaining it in the heavy traffic range. 

References 17 and 18 exhibited situations where state sensitive 
network behavior can be accommodated by diffusion models. It is well 
known that G/G models or state dynamic models are not amenable to 
exact analysis using traditional queueing theory approaches. 


Il. COMMENTS ON CONVERGENCE TO DIFFUSIONS 


In this section we very briefly describe the convergence of Q(nt)/ 
Vn to q(t). We aim our presentation to highlight those aspects of the 
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convergence that provide the backdrop for our main result. The 
material in this section is tutorial. The abstract convergence theory is 
adapted from Refs. 19 through 23, while the queueing network specific 
results cited can be found in Refs. 7, and 24 through 26. 

We introduce some notation needed in the sequel. Superscripts, - 
unless they are simple fractions, indicate the nth term in a sequence 
rather than exponentiation. By max (q(t)) we mean that the maximum 
is over the time interval [0, T]. We use ~ to denote asymptotic 
equivalence for large values of a parameter (f(n) ~ g(n) if g(n)/ 
f(n) > 1 as n— ©). For example, n’/ + n™? logon = n¥? 


3.1 Convergence of random processes 


The set D¥ of all vector-valued functions on a time interval [0, 7], 
where the components have left-hand limits and are right-continuous, 
is a very general set of functions for applications. Included are the 
diffusion sample paths, which are continuous, and the network 
queueing processes, which have piecewise constant paths. In the set 
D® a definition of distance between functions has been given. The 
metric is too involved to be discussed here, but for our purposes we 
will only encounter pairs x; and x2 whose distance can be measured by 
p(x1, X2) = max|| x; — x2], where || || is the usual definition of length of 
an L-dimensioned vector. 

By Q* we mean the set of all vector-valued random processes whose 
sample paths are in D”. A notion of distance between processes has 
been defined so that meaning is given to the convergence (=) of a 
sequence of random processes to a limit process. The metric space J” 
is a highly specialized topic, and we shall not define or discuss it in 
detail. However, we mention facts about QZ” that are relevant to our 
result. 

QB" accommodates both network queueing processes and diffusions 
and provides the setting for demonstrating convergence of queueing 
processes to diffusions. Since D” is a metric space, it makes sense to 
consider continuous real functions defined on D”. If f is a bounded 
continuous real function on D¥%, then f evaluated on the paths of an 
element of Y” is arandom variable. Indeed, f maps random functions 
into random numbers. If {x”"}f is a sequence of stochastic processes 
(points in J”), then {f(x”)}? is a sequence of random variables. x” is 
said to converge to x if for all bounded-continuous f the distribution 
functions of f(x,) converge in the usual sense to the distribution 
function for x. This definition of convergence of general*vector proc- 
esses is considered to be the definition because of its significant 
practical value. This practical value partially stems from the fact that 
all the finite dimensional distributions of g”(t) = Q(nt)/ Vn converge 
to those for q(t). Even more important is that the continuous functions 
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on the processes include those of interest in practice, and we have 
already mentioned that, by definition, the distributions of continuous 
functions converge. For example, a chief concern in the operation of 
a network is that queues not exceed certain levels (corresponding to 
overflow) or that maximum delay not be excessive. Thus, there is 
interest in max,Q"(t), and the maximum over a time interval can be 
shown to be a continuous function. Therefore, in situations where 
q"(t) is intractable and q(t) is tractable, it is meaningful to use q(t) to 
approximate the behavior of qg"(t) for large n. Regarding tractability, 
we note that a great deal is known about Wiener processes, so that 
properties of general limits of queueing processes can often be easily 
obtained from previously derived results. 


3.2 The G/G/1 queue 


We shall need this example in Section IV. For arrivals the mean 
rate is \ per second, while a denotes the interarrival variance. For the 
successive service times yu’ is the mean and s is the variance. 

Let Axu(71, 72) denote the number of arrivals in (7;, 72), and let 
D:(71, T2) denote the number of departures. Define 


Ax(0,nt) — Ant 
ae as : 


Using the central limit theorem and an asymptotic (large n) analysis, 
it is not difficult to show that a”(t) is distributed as N(0, A\°at). The 
connection to the central limit theorem stems from the fact that 
{Ax(0,nt) << Rk} = {A,; + Ao +--- A, > nt} (where A; are the interarrival 
times). 

We can conclude the same sort of result for departures, but that is 
more difficult. Consider first an imaginary system where the departure 
process runs forever with no idle times. Then, for each ¢, 


a’(t) = 


d"(t) = er is distributed as N(0, pst). 
n 


For this imaginary system a”(t) — d"(t) is distributed as N(0,(u°a + 
u’s)t). These elementary results regarding a”(t) and d"(t) are sugges- 
tive of much more significant results regarding convergence in J'. 
For the D' convergence, we can let » and X be functions of n (and 
write «” and X”) so long as uw” and dX” converge to constants and to 
each other so that 
lim Vn (7 — w") = A 


n—oo 


is a constant. This flexibility in u” will be useful in Section 4.2. For 
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each n, we have an arrival sequence {Aj;,}7 and a departure sequence 
{B,j}7 of i.i.d. random variables. For the imaginary system 


an(p) = nt) _ Aw(O,nt) — Dy(O,nt) 
: Jn Jn 


_ (Ax(O,nt) — A*nt) + (Da (O,nt) — wnt) " (u” — Xd”) E 
Vn vn 


we have that g”(t) converges to N(At, \°a + y’s). The convergence is 
for each t. But much more is true, namely, it has been shown that 
q"(t) converges to W(At, \°a + ws) in D'. Now for q"(t) one needs to 
account for departures not occurring when the queue is empty. This 
is a substantial complication that has been dealt with in Refs. 7, and 
24 through 26. The upshot is that q”(t) > W(A, d\°a + ps). 

For W (or W) one can write and solve a partial differential equation 
(called the Fokker-Planck equation) for the probability transition 
density associated with the system being in state q’ at time ¢ given 
that it was in state gé at an earlier time ft.”” So long as A < 0 the 
process W has a statistical equilibrium for which the state density is 
characterized by taking the limit of the probability transition density 
as > 0, 

In higher dimensions (networks) there is a vector version of the 
construction’ in which the boundaries present even more difficulties 
than in the one-dimensional case. 





t, 


3.3 Two results for future reference 


We next cite two additional results* from the theory that we will 
use to derive our result on communication of status. 


Lemma 1: If {x"}? and {y"}? are each positive sequences of random 
processes in D* with x” = x, then if, for each «> 0, 


lim Pr{max|| x"(t) — y(t) || = e} = Q, 
n—00 t 


we conclude that y" => x. 

Lemma 2: If {x"}ne1 is a sequence of random processes in Q" with 
continuous paths, convergent to a process with continuous paths, then 
for each positive « and n there exist a 6, 0 < 6 < T, and an integer no 
such that 


P{ max || x"(s) — x(t) {| = e} <7, n= No. 
|s—t|<6 


* Those familiar with the subject of convergence of probability measures will notice 
that Lemma 1 is a weakened form of the Converging Together Theorem (4.1 in Ref. 
20), where the Skorokhod metric p has been replaced by a simpler metric that dominates 
p. Lemma 2 is a tightness consequence (8.2 in Ref. 20). 
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IV. DERIVATION OF THE MAIN RESULT 


4.1 Approach 


Refer to Fig. 1. Were it not for the complication of the status packets 
on system 2, we would simply have a pair of independent G/G/1 
systems. The queueing process (qi, q3) would converge in J? to a 
two-dimensional W process with each independent component a W 
process in Y', as already described in Section 3.2. We want to show 
that in the limit the status packets adequate to describe q,(¢) in real 
time can be communicated on system 2 without perturbing the J’ 
limit, go(t), from that for the case with no status packets. 

So far we considered T' to be fixed, and consequently, we have 
written Q” rather than use the more complete notation D* [0, T]. 
The variable T is needed to express mathematically the meaning of 
communication of status in real time. We mean that for each time T, 
from the status packets communicated up to time T, we can construct 
a sequence, r”(t), so that for each e > 0 


lim Plo(r"(t), qi(t)) > J = 0 


(i.e., the distance between the process qi(t) and its approximation 
converges in probability to zero). From Lemma 1 we see that for each 
T the convergence of r”(t) to qi(t) in D? [0, T] is implied. 

In Section 4.2 we derive a channel for status information that is 
asymptotically, for large n, of negligible rate in comparison to the total 
information rate of the communication resource. In the remaining two 
subsections, it is shown that the changes in q7(t) are such that there 
is enough rate to convey a sampled, clipped version of q{(t) that 
satisfies the above requirement for r”(t). The purpose of the clipping 
is to limit the number of bits per sample so as to meet the capacity 
limitation on the status channel. 


4.2 Deriving a channel of negligible rate for status 


The normalization parameter n can have different interpretations, 
as discussed in Section II. For definiteness in the presentation of the 
proof, we choose the perspective that the demand for service, A”, is 
increasing and the capacity of the resource, M”, is likewise increasing: 


coale-d) 


M”" = mn, 


and 


where p and A are constants. 


PACKET NETWORK 473 


From this viewpoint we see countervailing aspects of the nature of 
the limit. On the one hand, with M” increasing indefinitely, we see 
the opportunity of deriving a status channel of some significance that 
is vanishingly small relative to M”. On the other hand, despite the 
fact that the limit q,(t) is insensitive to considerable changes in 
Qi(t), the limit has an extremely intricate structure that must be 
conveyed on the status channel. In fact, the w(t) constituent of q,(t) 
is the epitome of a chaotic continuous random function—it is the 
indefinite integral of white Gaussian noise.” 

Proposition: There exists a channel of negligible rate for status. 
Proof: The units of A” and M” are packets per second. Say that there 
are on the average B bits per packet so that the communication 
resource has a capacity of M”B bits per second. Since M"/n and A"/n 
correspond to »” and X", respectively, as indicated in Section 3.2, we 
can alter M” and not perturb the q,(t) diffusion if we preserve the 
asymptotic behavior: 


M”" — A" 
vn 


M"/n— pw» and — A 


for large n. 
This flexibility in M” enables us to derive a status channel. Specif- 
ically, we choose a channel for status that has rate 


1 
nis (; + | logon 


bits per second, where @ is any positive number. We shall see the 
adequacy of this rate. If the transmission resources for channel number 
2 provide the status channel, then 


M” = np — {n*"[(1/2) + 6]logon}/B 


packets per second, and still M” > np and 


(5+ 6) login 
2 A 
eT b-8) 


B vn 


M"— Av =n — n'? 


1 
ni8 (3 + | logon 


2 
=vnA- 7 


From Section 3.2 we have that q3(t) converges to the same limit 
whether or not the server provides the derived channel. 
We use the derived channel as follows to transmit information for 
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the required approximation to Q7(t): first we clip the process Q7(¢) at 
the upper threshold n’*"/”), and then we sample the result every n=? 
seconds. So, the set {0, 1, --- [n’*"/]} is the range of the samples 
(where [n’*“/?)] means the largest integer less than n?*"/”’), 

We now show that in the limit of large n the clipped and sampled 
replica of Q7(t) has the desired convergence property. Partition [0, 7] 
letting (i, n) denote the interval [(i — 1)/n*”, i/n’*], i= 1, 2, ---. In 
what follows, let v"(t) denote the piecewise constant process defined 
by qi(i/ Vn) on I(i, n). Let r”(t) be defined in precisely the same way 
as u"(t) except that r”(t) is truncated at n° so that r"(t) = 
min(n°, v"(t)). (Of course, if qgi(t) has an upward or downward drift, 
it would seem advisable to define v”(t) and r”(t) to have a linear slope 
between samples matching the trend in q{(t). However, the piecewise 
constant r”(t) and v"(t) are adequate for us and connect easily to 
existing results we want to use.) The successive values of r"(t) are 
transmitted over the derived channel. The function v"(¢) is introduced 
to help establish the convergence of r”(t) to q,(t). 

Using the above proposition we can now establish convergence. We 
must demonstrate the following. 


Theorem: For each « > 0 
lim Pr{max| qi(t) —r"(t)| >. } = 0. (1) 
n—-0 t 


Proof: It is convenient to note two ways in which r”(t) can fail to track 
qi(t) to within e: 
A. qi(t) could take on large values sufficiently beyond the peak value 
n° of the tracking process. 
B. q(t) could, on some I(i, n), depart too much from the sample value 
approximation. 
We shall see that both ways occur with sufficiently small probability. 
It is evident that 


{max|qi(¢) — r(t)| > 


is contained in 
{max q(t) > n°} U {max| q(t) — v"(t)| > ef = A” U B’, 
t £ 
where A” and B” are defined in the obvious way. We next show 
lim,_..P{A”"} = lim,_..P{B”} = 0. From these limits (1) follows. 
To see that P{A”"} — 0, use the fact that g{(t) = w(t) so that 


max,qi(t) — max,w(t). In terms of distributions we have that for each 
number y 


lim P{max gi(t) > y} = P{max w(t) > y}. 
n—0 t t 
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Now 
lim P{max qgi>nt< lim Pimax qi(t) > C} 


n—-0 


= P{max w(t) > C}, (2) 


where C is any number. So the right-hand side of (2) can be made 
arbitrarily small. Thus we have obtained the desired result for A”. 
Next we show that lim,_...P(B”) = 0; that is, for each « > 0 


lim P{max| q(t) — v"(t)| = e} = 0. (3) 


First let gi(t) denote the continuous variant of q{j(t) formed by 
connecting the consecutive points of discontinuity in the graph of 
qi(t) by line segments. By construction max,| q7(t) — q7(t) | < (1/ Vn). 
So by Lemma 1, g7(t) > q(t). Employing Lemma 2, we have that for 
each 7 > 0 there exists a 6 > 0, so that for 


P{ max | G7(t) — Gi(s)| = <7 
[s—t|<6 


for sufficiently large n. Therefore, 


P{ max, plait) — G(s) | = <n 


|s-t}]< 


for sufficiently large n, or what is the same, the limit of this sequence 
of probabilities is zero. Rewriting the maximum in terms of q,(t), we 
have 


Ple"+ max, Jat(t) — gi(s)| = J > 0, 


where the magnitude of the error term, e”, cannot exceed 2/ Vn. Since 


limP{ max, |ai(t) — qi(s)| = — en} = 0, 


n—-0 |s—t|<n 
by set containment it follows that for each fixed e«’ > e 


P{max| q(t) —v"(t)| = e} — 0. 
t 


The containment stems from taking the maximum over a smaller 
set and from the fact that the threshold e’ eventually exceeds « — e”. 
Now (8) follows because « is arbitrary. 


V. DISCUSSION 
5.1 Immediate extension 


We have proven a simple form of the result that in heavy traffic, 
status can be conveyed unintrusively. The result extends immediately 
to much more general network settings. It is not difficult to go beyond 
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the simple form of the result and conclude that status can be conveyed 
unintrusively throughout an entire network such as Reiman and 
Harrison’s generalization of a Jackson network. One simply derives as 
many parallel status channels as is necessary on each communication 
link. 


5.2 Similarities 


While heavy traffic analysis of computer communication networks 
is more intricate than established analytical techniques for electrical 
networks, there are some striking parallels. For example, as with 
electrical networks, one writes a differential equation’ (the aforemen- 
tioned intricacy stems from the fact that in computer networks it is a 
partial-differential equation: the Fokker-Planck equation). There is 
interest in both transient and steady-state analysis. As with voltage 
and current there is an extremely simple steady-state relationship 
between the two dependent variables of most interest, queue size, and 
delay.””° The main result of this paper can be considered to add to 
this list of similarities by deriving what amounts to a sampling 
theorem. 


5.3 Future work 


The availability of status information that was established here is 
only one aspect of the broader subject of network control. This 
availability prompts the question of how should status information be 
used to optimally control g(t) under certain performance criteria? 

Examples already appear in the literature’”® that demonstrate 
significant improvements using status information to guide the evo- 
lution of a queueing network in heavy traffic. One of the established 
examples involves that of the case of a Poisson arrival process where 
each arrival has the option of joining one of K queues. The K queueing 
systems have i.i.d. exponential service times. Two systems are con- 
trasted. In the first, status information is used and the arrival joins 
the queues offering the least expected delay at the time of arrival. In 
the second system the arrival is blind to status information and 
randomly selects a queue. In heavy traffic the performance of the first 
system is superior to that of the second by a factor of K in mean queue 
size and delay and, more importantly, a factor of K in the tail exponent 
of the equilibrium distribution for queue size and delay. 

The published examples seem part of a more general theory that 
would allow more relaxed assumptions on arrivals and services and 
would address the question of how status information should be 
transferred and used. The result of this paper needs to be shown in 
the context of g(t) being controlled with the status information. We 
remark that the proof we have given utilizes very little of the structure 
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of the approximating queueing processes and appears to allow great 
flexibility of form for Q”(t). 

For a Reiman-Harrison-type of network, but with state dynamic 
routing, the problem of finding the optimum control to minimize the 
maximum (over source-sink pairs) average delay is a very challenging 
one. 

Another layer of difficulty is introduced if one includes fixed delays 
in status information to account for processing or propagation. 
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A Note on Discrete Representation of Lines 


By M. D. McILROY* 
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In raster graphics a line must be drawn as a “discrete segment”, a set of 
integer grid points that lie close to the line. Equivalence classes of identically 
drawn lines are described in terms of Farey series; this treatment notably 
simplifies previous work of Dorst and Smeulders. A log n algorithm serves to 
identify a line’s equivalence class. Using it to choose among precomputed n- 
pixel discrete segments, we may draw lines in n-pixel blocks rather than the 
customary single-pixel steps. 


I. INTRODUCTION 


The problem of approximating a straight line by points of an integer 
grid may be reduced to that of finding the extreme grid points in a 
closed half-plane.” By reflections and translations the problem may 
be standardized to the form: For each integer x maximize y over the 
integers subject to 


y<mxt bd, 
where m and DJ are restricted to the region 
S={(m,b)|0<ms1,0<6b< Il}. 


An equivalent formulation is: For each integer x find the unique integer 
y that satisfies 


mx+b-1l<y<mxt+b. 


An explicit solution is y = Lmx + bJ, where Lb] denotes the largest 
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integer that does not exceed b. The conventions of the preceding 
equations will be followed throughout: 


n, DP, q, xX, y are integers, 
s, t, u, v, w are rationals, and 


b, m are reals. 


In particular, range restrictions, such as 0 <x <nand0<m &€l, 
must be understood in the domain of the variables involved. 
A discrete segment of length n is a set L(m, b, n) given by 


Lim, b, n) = {(x, Lmx + 61) | O<x <n}. 


Usually n is fixed, so we may speak simply of a discrete segment. 

Section II investigates the equivalence classes of lines that are 
approximated by identical discrete segments. These equivalence 
classes induce an interesting polygonal pattern in the space of param- 
eters (m, b). Farey sequences abound in the pattern, which is accord- 
ingly dubbed a “Farey fan”. To each region, or “facet”, of a Farey fan 
there corresponds a distinct discrete segment. Thus the facet in which 
the parameters of a given line lie tells what discrete segment approx- 
imates that line. A canonical characterization for facets and an 
O(log n) algorithm for locating the facet for a given line are developed 
in Section III. The algorithm may be used in raster graphics to choose 
among precomputed discrete segments, so that lines may be drawn by 
block transfers in n-pixel chunks. The main results were first given 
by Dorst and Smeulders.? The contributions of this paper are (1) 
recognition of the role of Farey series, (2) simplified analysis based on 
the Farey fan, and (3) algorithms. 


Il. FAREY FANS 


To each discrete segment L of length n there corresponds an 
equivalence class C(L) of points in S given by 


C(L) = {(m, b) | (m, b) € S and L(m, b, n) = L}, 
or 
C(L) = {(m, 6) | mx +b-—1<y< mx +t), (x, y) € LZ}. 


Defined by linear inequalities, the equivalence classes are polygonal 
subregions of S, called facets. For each point (x, y) in L there is a pair 
of parallel bounding rays, R(x, y) and R(x, y + 1), where 


R(x, vy) = {(m, 6) | b = —xm + y}, O<ySxSn. 
R(x, y) has slope —x, b-intercept y, and m-intercept y/x. The m- 


482 TECHNICAL JOURNAL, FEBRUARY 1985 





Fig. 1—Farey fan of order 6. Heavy lines delimit region S. 


intercepts {y/x | 0 <y <x, 1 <x <n} constitute a Farey series.* (See 
Chapter 3 of Ref. 4.) The rays make a pattern like Fig. 1. The part of 
this figure that is contained in the closure of S is called a Farey fan of 
order n, often shortened simply to “Farey fan”. 

A duality holds between the m-b space of Fig. 1 and the original 
x-y space where we are approximating. To every point (x, y)' in original 
space there corresponds a line b = —xm + y in dual space, whose 
points (m, b) are the slope and intercept parameters of members of a 
pencil of lines through (x, y) in original space. Similarly, to every point 
(m, b) of dual space there corresponds a line y = mx + 6 in original 
space, whose points (x, y) are slope-intercept parameters of members 
of a pencil of lines through (m, b) in dual space. 

A side of a facet is either a side of S or a segment of a ray. To 
include the top and bottom sides of S in the Farey fan, we admit 
R(0, 0) and R(0, 1) as well: 


R(x, y) = {(m, b) | b = —xm + y}, { 


* The Farey series of order n is the ordered set of rational numbers p/q, 1 < q <n, 
in the interval [0, 1]. For example, the Farey series of order 7 is: 


0111121232143 253 46561 


PU CS TS ee e687 A 8 6 aT: 


t In this paragraph x and y are real variables. 
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The vertex where two rays, R(x, y) and R(x’, y’), x > x’, intersect is 
the point 


x— x’? x—x’ 


(m, b) = Qs a=) ; (2) 


To simplify the analysis further, we extend the domain of R(x, y) 
again, to include 0 < x < n and all integer y. None of the extra rays 
passes through S, so this extension has no effect on the facets. In the 
strip 0 < m < 1, the m-coordinates of the intersections of R(x, y) with 
other rays are then given by rational numbers 


m=", 0O<psq, 1<q < max(x, n — x). (3) 


Taken in arithmetic order, these values constitute a segment of a 
Farey series of order max(x, n — x). From (1) any ray R(x, y) must 
cross m = p/q at b = y — px/q, that is, at a multiple of 1/g. We have 
proved 


Theorem 1: The abscissas of intersections of a given ray of a Farey fan 
with other rays constitute a Farey series. 


Corollary 1: The abscissas of adjacent vertices of a facet are adjacent 
members of a Farey series. Moreover, if the abscissa of a vertex is p/q, 
the ordinate is a multiple of 1/q. 

Figure 1 suggests that every facet is triangular or quadrilateral. This 
observation is true in general: 


Theorem 2: A facet of a Farey fan has at most four sides. 


Proof, by induction: The statement is obviously true for a Farey fan 
of order 1. Assume it true for Farey fans of order n — 1, and assume 
also that all four-sided facets are diamonds that have two middle 
vertices with identical m-coordinates. Recalling that all rays have 
nonpositive slope, we see that each facet of the Farey fan of order 
n — 1 has one of the shapes exemplified in Fig. 2. Each shape will be 
considered separately. 

Figure 2a shows a triangular facet bounded by m = 0. The only new 
line that crosses (enters the interior of) the facet in question is as 
shown in Fig. 3a. It divides the facet into two triangles. 

. Figure 2b shows a general triangular facet with long side upward. 
Any new ray that crosses the facet has a slope more negative than 
that of any of the sides and hence must cross side uw (Fig. 3b). Now 
u and w are adjacent members of a Farey series of some order g. From 
eq. (3) u, s, w must be members of a Farey series of order g + 1. Since 
at most one new member can appear between any two adjacent 
members of a Farey series in going from order g to order g + 1, at 
most one new ray can cross side uw. Thus u, s, w are consecutive 
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(a) (b) (c) (d) (e) 


Fig. 2—Possible facets of a Farey fan: (a) triangular facet bounded by m = 0; (b) 
general triangular facet with long side upward; (c) diamond facet; (d) general triangular 
facet with long side downward; (e) triangular facet bounded by m = 1. 


\ 
(a) c) (d) 


Fig. 3—Adding lines to a Farey fan of order n — 1 to make a Farey fan of order n. 
Vertices are labeled with their m-coordinates. 
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members of some Farey series. By Corollary 1, there is, for each pair 
drawn from {u, v, w}, a Farey series in which that pair is adjacent. 
Hence u, v, w are also consecutive members of some Farey series. 
Consequently s = v. The facet has been partitioned into one triangle 
and one diamond. 

Figure 2c shows a diamond facet. A new ray that crosses two opposite 
sides as in Fig. 3c would join points s and t, where s < v < t and where 
v belongs to a lower-order Farey series than do either s or t. Thus s 
and ¢ cannot be adjacent members of any Farey series, contradicting 
Corollary 1. A ray that crosses two adjacent sides (Fig. 3d) would 
introduce new terms into both Farey series. Since the flanking terms, 
u and v, are the same in both series, the new terms must be the same: 
s = t. But this is impossible because the new ray Would have infinite, 
not negative, slope. Because a new ray that crosses a diamond can 
cross neither two opposite nor two adjacent sides, it must pass through 
one of the two middle vertices and partition the diamond into a triangle 
and a diamond. 

Figures 2d and e are analogous to 2b and a. 

In every case the addition of rays to convert a Farey fan of order 
n— 1 into a Farey fan of order n yields triangular or diamond facets. 
The induction is complete. 


Hl. IDENTIFYING FACETS 


Since distinct discrete segments of length n correspond one-to-one 
with facets of the Farey fan of order n, the Farey fan makes a natural 
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Fig. 4—A ladder from Fig. 1. 


index to discrete segments. As Sproull has suggested,” one may use 
such an index to generate arbitrary approximate lines in gulps of 
length n, possibly beating the usual point-by-point methods. The 
words of a display memory make natural gulps. 

To look entries up in the index, we need a way to identify in which 
facet a given (m, b) pair lies. Let f; be the ith term of the Farey series 
of order n. Since all ray intersections in the Farey fan occur at points 
whose m-coordinates belong to the series, in each interval f, < m < 
fi+, the fan is a simple ladder of rungs that never cross each other in 
the interior (Fig. 4). This observation suggests Method M below, a 
workable method for locating the facet in which a given (m, b) lies. 
Method M: 

1. Find the interval to which (m, b) belongs. 

2. In that ladder locate the highest rung that lies on or below 
(m, b). 

3. From that rung read off a precomputed discrete segment of length 
n. 

An easy way to do Step 1 is binary search in the Farey series, or, 
since Farey series are fairly smooth, interpolation search. Step 2 can 
be done by binary search in the ladder. The total running time is 
O(log n): Step 1 searches a table of O(n”) Farey series members (see 
Ref. 4, Theorem 231); Step 2 searches a table of n rungs (Theorem 3). 

The suggested implementation of Method M requires preprocessing 
and storage for O(n”) intervals times n rungs of tabulated information. 
This information is dispensable. Step 1, identification of an interval, 
can be done by an algorithm of Papadimitriou in O(log n) time.® The 
equation of a rung through (p/gq, r/q) for use in Step 2 can be generated 
by substituting (m, b) = (p/q, r/q) in (1): 


Qy — Px = 7, 


and solving for x and y by an extended Euclidean algorithm in 
O(log n) time.” One solution is required at each of the O(log n) stages 
of binary search. Thus facets can be identified in O(log? n) time, single- 
shot, with no preprocessing. Preprocessing, however, reduces the prob- 
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lem to simple binary search; it is the technique of choice for modest, 
fixed values of n. 

Any scheme for uniquely identifying facets may be used for a 
canonical description for discrete segments. Before we demonstrate 
one, it will be helpful to have another theorem. 

Theorem 3: A ladder in a Farey fan of order n has exactly one rung of 
each slope —x, 1 < x <n. Moreover, adjacent rungs that meet at a 
ladder edge, m = p/q, where gcd(p, q) = 1, have slopes that differ by q. 
Proof: By eq. (1) a rung of slope —x must meet the vertical line 
m = m, at an ordinate b such that there exists an integer y satisfying 


0<b=-xm+y< 1. (4) 


Equation (4) reflects the fact that S is open along the top edge, b = 1. 
The closure of a rung, however, may meet that edge; we replace < with 
< in (4) to capture such rung ends. The solution 


y = [xmol (5) 


satisfies the constraints of (1) since 0 < mp < 1. It is unique unless 
xm = 0, in which case y = 0 and y = 1 are both solutions. But only 
one of two parallel rungs incident on (mo, 0) and (mp, 1) actually lies 
in S. The first claim of the theorem is established. 

To prove the second claim, let two rungs R(x, y) and R(x’ y’), 
x > x’, meet at m = p/q. From eq. (2), 


ey ae 
q Xx—-%X 





showing that q | x — x’. Thus we must have 


y=y'+khp, x=x' + kq (6) 
for some positive integer k. Substitution in (2) yields 
» = a Pe 
q 


Since this value of b is independent of k, all rungs R(x’ + kq, y’ + kp) 
meet R(x’ y’) at the same point. If any value of k in (6) yields values 
of x and y that satisfy (1), then k = 1 certainly does. Thus, if —x’ is 
the larger slope of two rungs incident on a vertex at m = p/q, there is 
another rung through that vertex with slope —(x’ + q), and no rung 
with an intermediate slope. The second claim has been proved. 

We now give the Dorst-Smeulders characterization of a discrete 
segment. Here the notion of “middle vertex” is extended from dia- 
monds to all facets, meaning.any but the upper left and lower right 
vertices. 
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1,1 


6,3 


1/4 1/3 2/5 1/2 


(a) (b) 


(c) (d) 


Fig. 5—(a) A fragment of Fig. 1. Certain rays are labeled with negated slope x and 
intercept y. (b) through (d) Frontier positions of lines in original space that are dual to 
facets labeled (b) through (d) in (a). The coordinates of breakpoints (@) are the same as 
the labels of the rays in (a) that are dual to the breakpoints. The discrete segment (X) 
for each facet is shown shifted up two units. 


D-S Representation: A facet of a Farey fan and its corresponding 
discrete segment may be characterized by four integers, n, q, p, and x, 
where 


n ts the length of the discrete segment, 
p/q, where gcd(p, q) = 1, is the abscissa of a middle vertex, and 


—x is the slope of the lower of the two rungs incident on the lower 
right vertex. 


Theorem 3 justifies the representation: the slope, —x, of the lower 
rung serves to distinguish among the various facets that have middle 
vertices on m = p/gq. 

The four parameters have simple interpretations. Each facet is a 
convex combination of its vertices, which represent limiting positions 
of lines in the original space. Figure 5 shows the lines in those limiting 
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positions; they form broken-line upper and lower frontiers. Any line 
that falls between the frontiers and does not touch the upper one will 
be approximated by the same discrete segment. If a frontier has two 
breakpoints, the slope of the line joining them is p/gq. 

Exactly one line of slope p/q can touch the lower frontier. Parameter 
x specifies the x-coordinate of the leftmost point where it touches; by 
(5) the y-coordinate is [ px/q1. This line, with slope p/q and intercept 
[ px/q| — px/q, may be taken to be the canonical representative of the 
equivalence class of all lines approximated by the same discrete 
segment. Because the canonical representative has rational slope, the 
discrete approximation to its infinite extension is invariant under 
translation by the vector (q, p). Hence the first differences (“chain- 
code,” in computer graphics parlance) of the discrete approximation 
are periodic with period g. Dorst and Smeulders took this periodicity 
to be the defining property of q. 


IV. DISCUSSION 


Rob Pike first brought the motivating question to my attention: 
How may one quickly identify the discrete segment of length n that 
approximates a given line? The apparent complexity of a quick sketch 
of the Farey fan discouraged further consideration—a cautionary 
experience. In this subject one well-drawn picture is worth a pot of 
algebra. In fact, computer graphics helped prove Theorem 2: the 
induction hypothesis came from playing with Theo Pavlidis’s PED 
graphics editor to make a Farey fan.® The conclusion of Theorem 2 
fairly shrieks from Fig. 1, and Theorem 1 isn’t far behind. In the 
absence of well-drawn diagrams, however, it took the work of Dorst 
and Smeulders to reveal the basic simplicity of the diagram. Their 
treatment involved intricate discrete analysis in the original space. 
The route through dual space, signposted by Farey series, turns out to 
be much easier. Moreover, the newer analysis solves the original 
problem: Method M may be used to select, from among a precomputed 
list, the correct discrete segment to approximate any given line. The 
method provides an exact alternative to a heuristic proposed by 
Sproull.° 

The search scheme in Method M may be recognized as a variation 
on Shamos’s slab test for the inclusion of a point in a polygon.””° The 
asymptotic performance of Method M is the same as that of Lipton 
and Tarjan for locating a point in a general polygonal tiling.'’ The 
single-shot algorithm, at O(log? n), is much better than general single- 
shot methods, which require O(n?) time in this special case.’° 

I am indebted to Jon Bentley for pointers from his mental encyclo- 
pedia of computational geometry. 
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This paper presents a model for designing the architecture of a distributed 
computing system of the type used to support the management and control 
activities of a large corporation. The input to the design process consists of 
two major data components describing the inputs and outputs to the infor- 
mation system and the relationships between them. Based on this information 
and the structure of communication and processing costs, an optimization 
model is formulated. It aggregates transactions in distributed databases, selects 
the locations in which those databases will be placed, assigns data sources to 
those databases, and selects for each report the report generation location. 
The problem is formulated as a combinatorial optimization problem and 
procedures are developed for computing lower bounds on the value of the 
optimal solution and heuristics for generating good feasible solutions for the 
problem. The procedures were tested on several examples and have generated 
good initial designs. Computational examples are presented to design problems 
including organizations with a hierarchical structure. 


I. INTRODUCTION 


Providing data collection, storage, and retrieval capabilities to in- 
dustrial service and governmental organizations is a primary respon- 
sibility of management information systems and of operations systems 
in those organizations. Traditionally, such services have been provided 
by centralized computing systems, using large computers with cen- 
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tralized databases and communication networks. Due to the increased 
dissatisfaction of users with the quality of services provided by cen- 
tralized systems, distributed computing systems are evolving as supe- 
rior substitutes to many of those large centralized systems, for provid- 
ing data storage and retrieval services. Distributed computer systems 
offer many potential advantages to those organizations. Those advan- 
tages include: 

e Reduction in data processing costs by trading communications for 

processing costs 

e Easier and smoother introduction of new hardware and software 

into the system 

e Shorter response time 

e Direct user control over computing resources to allow for setting 

up their own scheduling priorities 

e Establishment of site autonomy and direct user responsibility and 

control over their application programs, and the quality and 
timeliness of the data that they generate and collect. | 

Distributed computing systems present system designers with many 
challenges, some of which already have been encountered in central- 
ized computing systems and were easier to solve in a centralized 
setting, while others are new and unique to a distributed environment. 
These include technical issues such as: system integrity; concurrency 
control;'* query parsing and decomposition;’ addressing, naming, and 
directory management; backup recovery;® and security.’ Other issues 
involve managerial and design problems that are faced by the managers 
and designers of those systems. These include: decisions on the allo- 
cation of files and databases to computer locations;®”” placement 
of processors and allocation of databases to those _proces- 
sors;’*© query optimization;’*1* directory assignment; and design and 
analysis of the communication subnetworks.’® 

The potential promised by distributed systems and the importance 
of the above questions have attracted the attention of many research- 
ers. Several models and procedures have been developed for answering 
some of those design issues. All of the existing models for file, database, 
or processor allocation assume that the physical and logical designs of 
centralized files or databases are given, and that the primary respon- 
sibility of the system designer is to assign those files to computer 
locations. 

The design issues are relatively well understood when the files and 
databases have been predetermined. Unfortunately, in reality, the 
situation faced by a distributed system architecture designer is inher- 
ently different. At the beginning of the design and analysis process, 
the files and databases are either not defined or, if they are given, they 
are expressed in terms that are recognizable and acceptable for the 
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design of a centralized computing system. If they are left unchanged, 
however, they could severely restrict the performance of a distributed 
system. This paper is a first attempt to define, formulate, and develop 
design tools for cases in which the structure of the databases is not 
given in advance. The underlying theme of this paper is the view that 
it is the responsibility of the analyst and system designer to determine 
the structure of the system databases and to allocate them to computer 
locations. 

The input to the design process consists, in part, of the set of 
locations (data sources) from which data are recorded, collected, and 
transferred to the system databases. The data stored in the databases 
are used in report generation processes to generate periodic and on- 
demand reports, or provide responses to queries. These reports are 
used by the companies that are served by the distributed computing 
system. The final destinations, volume, and generation frequency of 
those reports constitute part of the input to the system design process. 

The data that are stored in the databases arrive there as transactions 
that have originated at the system data sources. The transactions 
report on events that have occurred in those locations. 

Many reports are generated from data stored in the system databases 
and are distributed to end users located in different destinations. A 
report generation process might need as input data that are retrieved 
from several databases located in distinct locations. The retrieved data 
have to be sorted, aggregated, and edited in order to generate the 
requested report; this report will then be distributed to those who use 
it as part of their regular activities. Different reports might be gener- 
ated in different locations. The decisions about which location should 
generate each one of the reports are based on which locations hold the 
databases that provide input to the report generation process and on 
the final destinations to which the edited reports have to be trans- 
mitted. 

The problem that is being addressed in this paper is as follows. The 
designer is given detailed information on data sources, the volume of 
update transactions that originate in each one of those locations, the 
list of reports and queries that have to be generated, their frequency, 
the volume of output generated, and their distribution to final users. 
For each report, the designer is also given the set of update transactions 
needed to generate it, the location (data sources) from which those 
transactions originate, and their volume per production run. A clear 
distinction is made between the locations from which update trans- 
actions originate and the locations in which those transactions are 
stored as part of the systems database. 

The system designer must decide: 

e How many processors to have in the system. 
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e What constitutes a database, i.e., what aggregations of update 
transactions form each one of the system databases, and how to 
partition those databases to realize some of the advantages offered 
by a distributed system. 

e Placement of computers and databases to locations, i.e., which 
locations will be selected for placement of processing capabilities. 

e Assignment of data sources to database locations, i.e., to which 
database location will the users in each data source send their 
transactions. 

e Report generation locations, the location in which each report will 
be generated, from which databases the data will be retrieved, and 
through what distribution channels the edited reports will be sent 
to the end users. 

Many factors have to be taken into consideration when making 
those decisions. These include operational restrictions, such as re- 
sponse time, reliability, availability, and security constraints; and 
many cost factors, such as: 

e Setup and operational costs for processors, storage devices, and 

access rights to communication channels. 

e The costs involved in transferring transactions from their origi- 
nation points (data sources) to their databases. 

e The cost of updating and maintaining the databases. 

e The cost of retrieving data needed for report generation from the 
databases. 

e The cost of transferring the retrieved data from the data retrieval 
locations to the locations in which the reports are generated. 

e The cost of generating a report in a location. Those costs might 
depend on the location in which the report is being processed. 

e The cost of distributing the edited reports to the end users, which 
could take the form of physical distribution of the printed reports 
through a carrier or mail service, or through the communication 
network. 

It is highly unrealistic to assume that in a single model we will be able 
to encompass all of those design issues. Instead we concentrate on the 
development of a simplified model that addresses the major cost factors 
influencing the design of a distributed computer system. The results 
obtained from such a model should be taken with caution and viewed 
as general guidelines for an initial design that can be used as input to 
the detailed design of such a system. 

An integral part of every systems analysis and design process 
includes the identification and definition of the sets of inputs (trans- 
actions) and outputs (reports) from and to the different entities of the 
managed system. The major objective of the design process is to decide 
on a data collection and report generation strategy that will minimize 


494 TECHNICAL JOURNAL, FEBRUARY 1985 


the system costs. To the best of our knowledge in spite of its practical 
and fundamental importance, this problem has not been addressed in 
the open literature, neither for centralized systems nor for distributed 
systems. 

In the next section we present the principal considerations, the 
setting under which the model has been developed, and the assump- 
tions used for a mathematical formulation of the problem. These are 
followed by a detailed description of the cost factors that affect the 
design of a distributed computing system. The problem is formulated 
in Section IV as a quadratic integer programming problem followed 
by a linear integer programming formulation of the problem. Several 
relaxations of the problem are introduced in Section V. The problem 
addressed here belongs to the class of NP-complete problems. In 
Section VI, heuristics are suggested to obtain approximate solutions 
to the problem. The heuristics and the lower bounding procedures 
were tested on several numerical examples and have generated good 
initial designs for those cases. Some of the computational results and 
a numerical example are presented in Section VII. In Section VIII, we 
present possible extensions of this model. The last section contains 
conclusions and suggestions for further research. 

The following terminology is used throughout the paper: transac- 
tions stand for update transactions, while reports stand for read-only 
transactions and include reports as well as query transactions. 


Il. UNDERLYING ASSUMPTIONS 


The problem of configuring a general-purpose large-scale distributed 
computing system is a complex task. Therefore we restrict the for- 
mulation and analysis by using simplifying assumptions. The following 
assumptions are used to formulate the distributed system architecture 
problem. In subsequent sections we will show how some of those 
assumptions can be relaxed and incorporated in the model, thus 
extending the range of cases that can be handled by this and related 
models. 

Assumption 1—(No splitting) All the transactions that originate in 
a data source are routed to the same set of computer locations. This 
assumption highly simplifies the directory for routing transactions 
from their origination location to the database in which they are 
stored. Under this scheme each data source has a unique set of 
addresses to which all its transactions are forwarded. 

Assumption 2—(No duplication) A single computer location and its 
associated database handle all the transactions that originate from the 
same data source. However, one or more data source locations can be 
assigned to the same computer. 

Assumptions 1 and 2 exclude the possibility of having multiple copies 
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of the same file in the system. The fact that the model is restricted to 
single-copy databases eliminates the need to consider the nonnegligible 
overhead required for the synchronization and concurrency control 
for updating multiple copies of the same database. This highly simpli- 
fies the analysis and modeling of the problem. In the last section of 
the paper we discuss situations in which multiple copies of databases 
are possible. 

Assumption 3—Databases can be assigned only to locations that 
contain computers with significant processing capacity. A distinction 
is made between I/O processors and processors whose main activity is 
arithmetic/logic operations. This implies that assigning a database to 
a location requires the assignment of a processor to the same location. 

Assumption 4—The processing of a report is done in a single 
location. All the data that are needed for processing a report are sent 
on demand to that location. This assumption does not exclude the 
possibility that only a fraction of the transactions will be sent from 
the database to the report generation location. This fraction depends 
on the selectivity and compression/aggregation of the data retrieved 
at the database location. The portion of data that is actually trans- 
ferred to the report generation location is not a collection of the 
original transactions. Typically it is only the compressed/aggregated 
content of the data selected from the databases. 

Assumption 5—Reports can be generated only in locations that 
contain computers. 

Assumption 6—Every report is generated in a separate report gen- 
eration process. The input, intermediate, or final results of one report 
are not used as input for the generation process of another report (in 
the same location or some other location). In reality it is possible to 
save on processing or communication costs by combining a few report 
generation processes into a sequence of report generation processes. 
By doing it, it is possible to eliminate duplication in the retrieval, 
transfer, or editing steps of the report generation process and reduce 
the systems costs. 

This assumption is clearly applicable to cases where reports can be 
initiated by users at any point in time. This eliminates the possibility 
of coordinating the report generation processes between the different 
users. However, it does not prohibit the possibility of combining the 
generation processes of reports that are always generated in the same 
run into a single report generation process. The model views the 
combination of such reports as a single report generation process with 
outputs being distributed to one or more final destinations. The only 
restriction that is imposed on such a combination is the requirement 
to specify beforehand the inputs and outputs from and to the combined 
process. 
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Using the above assumptions, four types of locations can be defined. 

D,—is the set of data sources. This is the set of locations from 
which data are collected and recorded to support the operations 
(decision making and control) of the organization that is being served 
by the distributed computer system. This is the set of locations in 
which people or automatic devices observe or detect changes in the 
state of the system; record them on forms or some other recording 
devices; and feed them through displays, data collection equipment, 
communication networks, optical scanners, or magnetic ink readers to 
the system databases. 

Examples of transaction initiators are bank tellers in a bank branch, 
window clerks in a warehouse, sales personnel in a marketing group, 
cashiers in a supermarket, or surveying and testing devices located in 
central offices generating automatic alarms or responses to equipment 
and facility testing requests in the telephone system. It is unrealistic 
to assume that the model developed subsequently will be able to 
capture this level of detail. Therefore a data source location is defined 
as an aggregation of all the transactions that were initiated from the 
same geographical area. This might be the set of transactions origi- 
nating from the same bank branch (which is an aggregation of the 
tellers in that branch), the set of banking transactions that originate 
from the same city/region (this is an aggregation of bank branches), 
the set of transactions that have originated in the same supermarket 
store, or all the transactions that originate in the same plant/labora- 
tory or warehouse. 

D,—is the set of places in which end users are located. Those users 
request reports that are based on data and transactions that have 
originated from locations in D,. Those are reports used by the head- 
quarters, regional managers, or clerks. Depending on the organiza- 
tional structure of the corporation, D2 can include locations that are 
identical to locations in D,, or locations that are not part of D;. 

D,—is the set of locations in which databases are stored. We assume 
that the existence of a database in a location implies the presence of 
a processor in the same location. 

D,4—is the set of locations at which reports are generated. This 
implies the availability of computing capacity in those locations. The 
intersection of D, and D3 typically is not empty. A location that 
belongs to D3 and D, is a location with a computer that maintains a 
database and is also used to generate reports for some users. 


Il. COST STRUCTURE OF THE PROBLEM 


The selection of a final configuration of a distributed computing 
system depends to a large extent on many cost factors. Some of those 
cost components are dependent on the amount of activity in the 
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system and could be nonlinear functions of the distance, number of 
transactions, or storage requirements. In this section we outline the 
major cost components that influence such a design. Distributed 
computer systems are expected to be operational over a long lifetime, 
with different expenses incurred at varying stages of the system life 
cycle. In order to bring those expenses into a common denominator, 
they are discounted to their present value, using appropriate discount- 
ing factors. 


3.1 Setup and operational costs 


These are the setup and operational costs of placing a computer and 
appropriate storage devices in a location. These costs are com- 
posed of purchasing/rental, installation, and maintenance of hardware 
and software; the physical facilities in which they are installed; and 
the staffing requirements for production runs, maintenance, and de- 
velopment activities. They include only cost factors that are inde- 
pendent of the amount of activity from and to that site, in terms of 
update transactions or retrieval requests. 


3.2 Data collection transfer and update costs 


Data collection transfer and update costs include: 

e The costs for recording as transactions events that occur at data 
source locations 

Routing all the transactions that originate in a data source to 
their database location 

Editing, verifying, validating, and updating the database with 
correct transactions 

e Sending a transaction update confirmation response to the trans- 

action originator and handling the errors discovered during that 
process. 
Those costs are composed of processing, communications, and storage 
costs, and depend on 

e The number of transactions processed over the planning period 

e The length (in bytes) of an average transaction 

e The duration in time that individual transactions are kept as 

active detailed on line data in the database 

e The amount of storage required to store the stable part of the 

database that contains data on the entities that belong to its data 
sources. 

The cost expressions make a clear distinction between the traffic 
intensity (measured by the number of transactions) from a given 
source to its database, and the amount of storage needed to store the 
relevant data pertaining to that source. The on-line storage require- 
ments for a single data source are composed of two components. They 
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are tightly coupled to the amount of data held on the different entities 
in the data source, and are also a function of the number of transac- 
tions held as active transactions in the database. 

Using a banking example, we can have bank branches that have 
relatively few accounts, but the average account holders in those 
branches are very active and therefore many transactions are gener- 
ated from and to those accounts. On the other hand, we can have 
branches that have many accounts, with very little activity going on 
in those accounts. In this specific example, there is very little relation- 
ship between the number of accounts (which are the dominant factor 
in determining the on-line storage requirements) and the average 
activity in those accounts (which determines the communication, 
processing, and update costs). 

The data collection, transfer, and update costs between points i and 
j are given by the following cost expressions: 


CH (Ai, EF) = f(A, EF) + f(A, E) + gf (Ai EF, 


where: 
A;—is the number of transactions per time period that 
originate from data source 1, 1 € J. 
E;—is the expected number of entities in data source 1, 

j € I for which data are kept in the system databases. 

f!?(Aj,E;)—captures the costs for data recording, collection, and 
handling of correct and rejected transactions, in data 
source i. These costs are primarily dependent and pro- 
portional to A;, in some cases it is also dependent on 
E;. 

fi (Ai, E;)—captures the cost of transferring transactions from data 
source i to a database in location J; this also includes 
the cost of sending responses to the transaction origi- 
nators on updated transactions, and the resubmission 
of transactions that have been rejected by the system. 
The expression f}})(.) consists mainly of communica- 
tion costs between points 1 and J, and is proportional to 
the number of messages transmitted and to the average 
message length. 

g\” (A;, E;)—captures the processing costs for editing, verifying, val- 
idating, and updating in location j transactions that 
have originated from source i. The expression g!(-) 
also includes the storage costs for storing in location j 
data that have originated or are relevant for processing 
transactions that have originated from data source 1. 
The storage costs depend mainly on the amount of 
storage needed to store information on entities in E£; 
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and to keep readily accessible in location j a subset of 
the transactions that are submitted from i to J. 


3.3 Data retrieval and transfer costs 


These include the costs for retrieving from a database in location J 
data that are needed for generating report r, r © R, and transferring 
the retrieved data to the location in which report r is generated. The 
data retrieval costs over the planning horizon depend on the frequency 
in which report r is generated and consist of: 

e Data processing costs for retrieving data from the databases, and 
sorting and aggregating the retrieved data at the data retrieval 
location (in order to reduce the amount of data that has to be 
transferred to the report generation location) 

e Data communication costs for transferring the retrieved data from 
the data retrieval location to the report generation location. The 
amount of data that has to be transferred depends on the selectiv- 
ity of the retrieved data (the ratio between the amount of data 
collected in the database and the portion that is actually retrieved 
for a specific report). In addition, the retrieved data are also 
aggregated, which further reduces the volume of data that has to 
be transferred from the system databases to the report generation 
locations. 

Using the assumption that those costs are separable over data 
sources, the following expressions give the cost of retrieving from a 
database in location j data that have originated in data source 1, ] € 
S,, are stored in location j, and are used as input to the generation 
process of report r, r € R. Given that report r is generated in location 
k, 


CE2(Ai, Ei, Sir) = £3 (Ai, Ei, Sir) + £2-(Ai, Ei, Sir) 


Si, = the selectivity factor between the amount of data 
entered into the database from data source 1, and 
the amount of data that is retrieved from the same 
database for report r. 

f\? (Ai, Ei, Si-) = the data retrieval and processing costs (in location 
J) for retrieving from a database in location J, data 
that have originated from source 1, 1 € S,, and are 
needed as input to the report generation process 
of report r. 

f2,(Ai,E;,S;-) = the communication costs for transferring the re- 
trieved data from the data retrieval location ] to 
the report generation location k. These costs are 
proportional to the amount of data that has to be 
transferred between the two locations (j and k), 
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and the characteristics of the data source i and 
the requested report r. 


3.4 Report processing editing and distribution costs 


These are costs for generating report r in location k and for distrib- 
uting the edited report to the end users of that report. They are 
incurred from the moment that all the data that are needed to generate 
report r are available at the report generation location (k). They are 
composed of processing costs in location k for combining the different 
data sets that were transferred to location k from database locations 
that contain data needed for report r. These data are sorted, aggre- 
gated, and edited to form report r. Communication costs are incurred 
when the edited reports are transferred to the end users. Since those 
cost components are not dependent on the locations from which the 
input data have been retrieved they are expressed as: 


Ci (A,, E,,N-) = fir (A-,E,) + gf (N,), 


where: 
A, = the volume of data that is used as input to report r. It 
is based on transactions collected from all the data 
sources that provide input for report r. 
E, = the amount of data that is related to all the entities 
whose data are being used as input to report r. 
N, = the set of locations that contain end users of report r. 


This is the set of locations to which the report will be 
distributed. 

f2@(A,,E,) = the generation costs (I/O, processing, and memory) 

of report r in location k. They are a function of 
A,,E, and the frequency with which report r is being 
generated over the planning horizon. 

g\(N,) = the distribution costs of report r, from location k in 
which it is generated, to the set of end users of this 
report. 

Large amounts of data have to be collected and analyzed in order to 
provide input to a system configuration model. This is a time-consum- 
ing and costly effort that might preclude the use of such a design tool 
owing to excessive data processing and preparation costs. Fortunately, 
under appropriate assumptions, some of those cost components are 
identical for different system configurations and are irrelevant for cost 
comparisons, and therefore need not be collected. For example, if the 
cost of generating report r does not depend on the location in which 
the report is being generated, then there is no need to specify 
f@(A,,E,) in the design model. Similarly, if the costs for recording, 
collection, and handling of data in data source i are independent of 
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the location in which those data are finally stored, then there is no 
need to specify the values of f!"(A;, E;) in C$} (Aj, E,). By paying careful 
attention to which cost components are included in the model, the 
amount of effort needed for data collection and analysis can be reduced 
significantly. 


IV. MATHEMATICAL FORMULATION OF THE PROBLEM 


In this section we present two mathematical programming formu- 
lations of the distributed system architecture design problem. The 
first formulation is an integer programming problem with a linear 
constraint set and a quadratic objective function. This class of prob- 
lems is extremely difficult to solve. Therefore, in the second formula- 
tion, the objective function is linearized by defining additional varia- 
bles and constraints. 

To formulate the problem, we define the following decision variables: 

Z; = abinary variable equal to one if a computer and database are 
placed in location j, and equal to zero otherwise. 

X;; = a binary variable equal to one if all the transactions that 
originate from a data source i are routed to a database in 
location j, X;;, and equal to zero otherwise. 

Y;,, = a binary variable equal to one if report r is assigned to be 
generated in location k, and equal to zero otherwise. 


The distributed system architecture problem is formulated as: 


Problem QIP: 
Find binary variables Z;, X;;, Y:, that satisfy: 


Zar = Min x Z;P; + »> », Xi Vij + >, > Yur Up, 


iE€l jEd rER ke 


sae ap ep ap) XiYnr Win} 


rER iES, jed ked 


subject to: ; 
jed 
Xi = Z; rElLjEed (2) 
»> Yor =] rER (3) 
kEed 
Vor S Zp rEeRkEed (4) 
Xij, Yjr, Zj) = O orl tELjEIrER. (5) 


The constraints in (1) ensure that all the transactions that originate 
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in data source 1 will be routed to exactly one of the locations that are 
candidates for database (and processor) placement; the constraints in 
(2) ensure that transactions will be sent from source i to location j 
only if a database has been placed in location j. Similarly, the con- 
straints in (3) guarantee that each report will be assigned to some 
report generation location, while the constraints in (4) prevent the 
assignment of a report generation process to a location that does not 
have processing capabilities. 

The objective function consists of the following cost components: 

P; = the cost of placing a computer and a database in location 
j. We assume here that placing a processor (or a database) 
in location j implies that the capabilities for handling a 
database (or report generation) have also been acquired. 
Those setup costs are incurred only when Z; = 1. 

V;; = the marginal cost over the planning horizon for sending all 
the transactions that originate from data source i to update 
a database that is placed in location j. This includes the 
communication costs for transferring the transactions from 
location 1 to location j; the transaction processing costs in 
location j for verifying, validating those transactions, and 
updating them in the database; storage costs for storing 
those transactions in the database; and the costs of sending 
rejection or acceptance responses from location j to data 
source i on the status of those transactions. Those costs 
are incurred only if X; = 1. 

U;,, = the marginal cost over the planning horizon for processing 
report r in location k. This includes the costs for editing 
the report in location k and for distributing its outputs to 
end users. U;, is incurred only when Y;, = 1. 

Wijzr = the marginal cost over the planning horizon of retrieving 
for report r, which is generated in location k, data that 
originated in data source 1, 1 € S,, which is currently stored 
in a database in location 7. This includes the cost of 
retrieving data from the database, selecting and aggregating 
it in location j and transferring the aggregated data from 
location j to location k, and preparing it as input to the 
process that generates report r (in location k). This cost is 
incurred only if the multiplication of X; by Y», = 1, ie., 
the transactions collected from data source i have been 
assigned to a database in location j, and report r has been 
assigned to a processor in location k. 

The mathematical programming formulation of Problem QIP is a 

quadratic integer programming problem. It has a linear constraint set 
and a quadratic objective function. The present state of the art in 
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solving nonlinear integer programming problems does not hold much 
promise of solving those types of problems in reasonable amounts of 
computing times. The objective function can be linearized by defining 
a new set of binary variables, V;;,,, rE R, iE S,,j7 Ed, RE dA. Vizx, is 
equal to one when report r is assigned to be generated at location 
k and it uses transactions that have originated from data source 
i,t € S,, and were stored in a database in location j. V;;,, is zero 
otherwise. 
The distributed system architecture problem is reformulated as: 


Problem ILP: 
Find binary variables Z;, X;;, Yur, Vijer that satisfy: 


Zip = Min 2 GP; +) X;V5+ ¥ XY VnUy 
(= 


ie€l jEed rER keJ 


PD de ds a Mie Wont 


rER jeJd keJ i€S, 


subject to: 
y; Xij =1 l E I (6) 
jEd 
Xi <= Zj ieEeLjed (7) 
> Ypr =1 rER (8) 
ked 
Yrr = Zp rER,keEd (9) 
Wijer S Xi; JEJAREIJrERIES, (10) 
WVijer S Y i JEJAREIJrERIES, (11) 
by > Wijer = 1 rE R,ieS, (12) 
jed ked 


Zj, Xi; Vins Wir = O orl 1€l1jEdJgkeEdJ,rER (13) 


The constraints in eqs. (6) through (9) have the same interpretation 
as constraints (1) through (4) in the quadratic programming formu- 
lation of the problem. The constraints in (10) ensure that reports that 
use data originating from data source i will be able to retrieve it from 
a database in location j, only if the transactions originating from 
location i have been assigned to update a database placed in location 
j. The constraints in (11) ensure that the data needed for report r are 
sent to location k only if report r is actually generated in location k. 
By definition, report r uses transactions that originate in location 
i,t € S,. These transactions must first be routed and update a database. 
When report r is requested, the data are retrieved from the database 
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and sent to the location in which report r is generated. The constraints 
in (12) ensure that such a routing to and from a database actually 
takes place for each combination of rE R andi € S,. 


V. GENERATING LOWER BOUNDS ON THE OPTIMAL SOLUTION 


The problem of optimally configuring a distributed computing sys- 
tem is a combinatorial optimization problem. It belongs to the class 
of NP complete problems. This statement can be proved by setting 
Wier = OVW1, j, k, r. The quadratic terms in Problem QIP are eliminated 
and the problem is reduced to an uncapacitated plant location problem, 
which has been proven to be in the NP complete class.”” This obser- 
vation implies that it is unlikely that an algorithm will be developed 
that is capable of solving every instance of the problem to optimality. 
Moreover, using similar arguments it can be shown that unless 
P = NP, it is not possible to develop a polynomial time algorithm 
which produces approximate solutions that have a guaranteed absolute 
error bound. The existence of such an approximation scheme implies 
the ability to generate feasible solutions that have an objective function 
value within a user-predefined error tolerance from the optimal solu- 
tion. Given this situation, the only alternative is for a system designer 
to resort to heuristics. Those heuristics can generate feasible solutions 
for the problem. However, they are not guaranteed to generate an 
optimal solution. Many heuristics have been developed for a variety 
of combinatorial optimization problems. The practical experience 
gained in using those heuristics is quite encouraging. Extensive com- 
putational experimentation with many of those heuristics shows that 
in many cases they generate solutions that are very close to the optimal 
solution. 

The solutions generated by the heuristics are feasible to the problem 
of configuring distributed systems, and as such provide an upper bound 
on the value of the (unknown) optimal solution. Since the value of the 
optimal solution to the system configuration problem is unknown, to 
evaluate the quality of the solutions generated by the heuristics, we 
develop in this section methods for computing lower bounds on the 
value of the optimal solution. The difference between the value of the 
best upper bound generated by the heuristic and the tightest lower 
bound generated by the developed method is clearly an upper bound 
on the gap between the value of the heuristic solution and the true 
optimal solution. Keeping in mind that the data supplied to those 
models contain errors that in many cases exceed the errors introduced 
by the heuristic, these heuristics seem to provide a plausible option 
for a system designer. 

We present two methods for computing lower bounds. The first 
method is based on a Lagrangian relaxation of Problem ILP, which, 
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together with a multiplier updating method, can produce tight lower 
bounds on the optimal solution value. This procedure requires exten- 
sive computational resources, which limits its applicability to cases in 
which the system designer is willing to incur those expenses. As an 
alternative to this time-consuming procedure, a second method is 
developed for computing lower bounds. This procedure generates lower 
bounds that are not as tight as the Lagrangian-based procedure, but 
are easier to implement and require only a modest amount of comput- 
ing resources. It is a preferable alternative for design problems in 
which a rough estimate on the quality of the solution suffices. 


5.1 The Lagrangian relaxation procedure 


The Lagrangian relaxation of Problem ILP is formed by multiplying 
the constraints in (12) by a vector {),-} of Lagrange multipliers, and 
the constraints in (11) with the vector {@;;,,}, and adding them to the 
objective function, forming the following Lagrangian problem: 


L(A, 6) = min 12 Z;P)+ DY XyVet+ YD Yee 


iE] jEd rER ked 


oy DD Wie Wie DD Ae ( ~y > vim] 


rER jéd ked iES, rER iéS, JEd ked 


Sse a AD Nem Mr te vin) 


rER jEd kd iS, 


subject to eqs. (6) through (10) and (18). 
After rearrangement and collection of terms that correspond to 
identical variable types, the objective function is reduced to: 


L(A, 6) = min x ZP,+ YY Xi Vij 


1€] jEd 


+ > > Yor (u.+ > 5, Bim) ry > Nir 


rER ked iES, jed rER i€S, 


+P DD D Wier (Ware — Bier — ra , 


rER jEJ ked iES, 


leading to the following optimization problem. 


Problem LR 
Find binary variables X;;, Yi;, Z;, Vijxr that satisfy: 
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L(d, 6) = XY Air + min 15 ZjPj+ YY XyVi 


rER ie€S, iEl jET 


+ » > Yor On + y, = b} 3 W ijn W, Wirt 
rER ked rER jed kEd iES, 
subject to the constraints in eqs. (6) through (10) and (13), 
where 
Une = Un a Y > Beis 


j€J ies, 


and 
Wir = Wie Bijer a Nir's 
Problem LR is relatively easy to solve, if we note that for a fixed 
vector X;; of X;; variables, the W;;,, variables in the Lagrangian problem 
must satisfy: 


: 0 if Xj = 0, 
W ijnr = J0if Xi =1 and Wijnr > 0, 
lif %,=1 and War <0, 


where W;;,, is the optimal solution to Problem LR for a fixed set of 
Lagrange multipliers. From the above relationship it is easy to verify 
that: 


Wijar Wijar = Xi min{O; Wijz,}. 


Using this relation, Problem LR can be rewritten as: 


L(r, 6) = 4 Y ZPi + 2 LY XyVe+ LD YO} 


1E] jE rER jEd 


(14) 
+min »} YA 
reR i€S, 
subject to: 

b> Xij — 1 1 E I, (15) 

JET 
Y Yj; =1 reER, (16) 

jEd 
Xi = Zj LE Ye E J, (17) 
Y, <Z; jEd,reER, (18) 
Z;, Xij, Yjr = O or 1 i€ljed,rER, (19) 
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where 
Vi = Vi + Y ¥Y min{0; Wir}. 


k&d rER 

The problem given by eqs. (14) through (19) is convertible to a 
simple uncapacitated plant location problem, in which the set of 
candidate plant locations is equal to J, and the set of customer 
locations is given by the union of the sets J and R. Uncapacitated 
plant location problems are optimization problems that belong to the 
class of NP complete problems. As such, it is unlikely that an algorithm 
that is capable of solving every instance of the problem to optimality 
in polynomial time will be developed. However, ample empirical evi- 
dence exists showing that several algorithms that have been developed 
by several researchers for solving uncapacitated plant location prob- 
lems can solve without too much difficulty many occurrences of the 
problem. Erlenkotter has developed such a code.”! It is a branch and 
bound dual-based procedure for solving plant location problems. The 
procedure was implemented and tested on a large set of cases and was 
able to solve without much difficulty many plant location problems to 
optimality. 

Given a fixed vector of Lagrange multipliers, it can be shown that 
L(A, 8) is a lower bound on the value of the optimal solution to the 
distributed system configuration problem, i.e., L(A, 8) S Zip. Different 
multiplier values will generate different values for L(A, 8). We are 
interested in obtaining a set of multipliers (A, 6) that generate the 
tightest possible bounds on Zyzp, i.e., they satisfy: 


L(A, 8) = max{L(A, 6)}. 
(\,8) 


Obtaining the optimal multiplier values is not a simple task. L(A, 8) 
is a nondifferentiable function, making a straightforward optimization 
of the multiplier values a difficult task. Several methods have been 
suggested in the literature for computing good approximations to 
(X, 8). Those procedures compute multiplier values that generate 
solutions that are very close to L(i, 8). Those procedures include 
methods such as column generation, dual ascent, multiplier adjust- 
ment, or subgradient optimization procedures.”””* 

The subgradient optimization procedure is an iterative method that 
starts with an initial set of multiplier values and uses a multiplier 
updating scheme to improve the lower bound value. The procedure is 
initiated with an initial set of multiplier values and uses the following 
multiplier updating scheme when moving from one iteration to the 
next: 


+1 
x NM, je LYE 
BPie = Bo = ype: 
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where d4, and 8%;,, are the multiplier values used in iteration p. The 
subgradient directions are given by: 
yee i py oy Wor, 
jed ked 

“phe = V5, aa Wr. 
Y%, and W?,, are the optimal variable values for the Lagrangian 
problem at iteration p, and ¢, is a step size. Poljack™ proved that the 
sequence (\?, 6?) converges to (A, 8) if the step size t, satisfies the 
conditions: 


lim t}=0 and } t=. 
p- p=1 
Unfortunately, such a sequence is not practical for a computer imple- 
mentation. Therefore, a heuristic is used for computing the step size 
tp. It is given by: 
Z — Lid’, 6°) 
ea ee 
where Z is an overestimate on L(), B) and 6, is a scalar in the range 
of zero to two. 6, is initially set to two and is divided by two if, in a 
sequence of n iterations, there was no improvement in the Lagrangian 
value. The procedure is terminated when one of the following termi- 
nation conditions is satisfied: 
1. The number of iterations has exceeded an upper limit. 
2. The difference between the upper and lower bound limits is below 
a given tolerance ¢, where 


tp = Op 


L(A, B) | 





3. The step size t, is small, ¢, < a. 

4, The scalar 6, is small, 6, S e. 

Extensive computational experience with the subgradient optimi- 
zation procedure reveals that it is not sensitive to the overestimate Z 
value, or to the initial multiplier values. The procedure is sensitive to 
the value of the parameter n, which determines the rate of change in 
the 6, value. Setting n to a high value implies that a large number of 
iterations will be needed before the procedure converges to a good 
solution. Setting n to a low value may lead to termination before the 
procedure had the time to converge. Appropriate n values can be 
determined only through computational experimentation. 

The combination of Lagrangian relaxation and subgradient opti- 
mization procedures has been successfully applied to a variety of 
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combinatorial optimization problems. Those include problems such as 
the traveling salesman problem,” multiple traveling salesman prob- 
lems,” topological design of computer communication networks,}*”* 
or routing in computer networks.’ The application of the same pro- 
cedure to the distributed system configuration problem is practical 
only for problems with a small number of reports and database 
locations. When applied to Problem ILP, the procedure requires the 
storage and update of (|J|? Yer | S,|) multiplier values, leading to 
large storage requirements and to a relatively slow convergence. 


5.2 A simple lower bounding method 


The Lagrangian relaxation-based procedure for computing lower 
bounds to Problem ILP is a computationally expensive procedure, 
which might preclude its use for many system design problems. An 
alternative method for computing lower bounds that requires signifi- 
cantly less computing resources, but is expected to produce inferior 
bounds, is based on the quadratic programming formulation of the 
problem. The method is based on the following argument. If the 
quadratic terms in the objective function of Problem QIP are dropped, 
the optimization problem is reduced to a simple uncapacitated plant 
location problem. The cost of this plant location problem is clearly a 
lower bound on the value of the optimal solution to Problem ILP. This 
bound can be further strengthened by adding to the U;, terms in the 
solution of the plant location problem an underestimate of the costs 
that have been neglected by eliminating the quadratic terms from the 
objective function. Those terms capture the costs for transferring from 
some unknown database locations to location k data that originated 
in source i that are needed for the generation of report r. 

Letting W;,, be the minimal cost of sending data that originate in 
data source i, i € S, to some database location, retrieving it from that 
location, and transferring it to location k where report r is generated; 
that cost is given by min| W,z-}. By summing up those individual costs 


over all data sources ‘that originate data for report r, a lower bound 
U,, on the data retrieval and transfer costs for generating report r in 
location k is obtained: 


Ur = Un + > min{ Wijzr} = Un + y Winer « 
ies, JE iES 


r 


Substituting Us, for Ur, in the relaxed problem leads to the following 
optimization problem. 


Problem YLP 
Find binary variables Z;, X;;, Y;, that satisfy: 
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2g = min | 5 Z;P;+ YY ZyVy + DY Yilied 


i€l jed rER jed 


subject to eqs. (15) through (19). 

This optimization problem is a simple, uncapacitated plant location 
problem. Its solution provides a lower bound on the value of the 
optimal solution. 


Theorem 1: Zin = Zar: 
Proof: Let (Z*, X*, Y*) be the optimal solution to Problem QIP, and 
(Z¥, XY, Y*) the optimal solution to Problem YLB. O 


Letting j; be the index of the database location to which data are sent 
and stored from data source 1, in the optimal solution to Problem QIP. 
From the definition of W;;,, it is clear that: 


Wij.tr => Wu = min{ Wir}. 
JE 


The quadratic terms in the objective function of Problem QIP satisfy 
the following relation: 


» » > y XE YE Wijrr 


rER jEJ keJd reES, 


= De x Wiier = LL YE 2d Wier. 


rER ked iES, rER ked ieéS, 


From the above relations it follows that: 
Zaire = Z*P + X*V + Y*U + X*Y*wW 


> Z*P + X*V + Y*O + y 1B Wich. 


Since (Z*, X*, Y*) is a feasible solution to Problem YLB, the following 
relation follows: 


Z*P+xX*V + YO + Y* 1 Wir} > ZYpP 
+0 + PO+ PY 4D Werle zy, 
iES, 


Thus Lip = Zarp- 
It is easy to see that this bounding method will provide a tight lower 
bound whenever 


>> max| Wii} K Uy reéER, RE, 


ie€S, jJEd 
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i.e., the marginal cost contribution of }) X;;Y:-Wijzr is small when 
compared to the other cost factors of the problem. 

By applying similar arguments to the X;; variables in Problem QIP, 
a second bound is generated by the solution to the following problem. 


Problem XLB 
Find binary variables Z;, Xi;, Y;, that satisfy: 


Zip = min x Z;P; + > > Xi Vi + » Dy Yun 
te] jET rER ked 
where 


Vii =V,;+ ¥ min {Wier}, 


réT; 


and T;, is the index set of the reports that use data from data source i. 
The lower bound value is given by the maximum of those two 
bounds: 


ZLB = max{Zi3; Zip}. 
The relationship between Zip and L(X, @) is established in the 
following theorem. 
Theorem 2: Zip < L(X, B). 


Proof: First we prove that Zig < L( r, B ). The feasible sets for Problem 
LR and Problem YLB are identical. Therefore, it suffices to show that 
the objective function of Problem YLB is a special case for Problem 
LR. The objective function for Problem LR can be written as: 


Is Z;P; 5 2 » > Xij Vij “TF >, > Yor (Une + oF Y Bijnr) 


i€l jeJ rER keJ iES, jEJ 
+ hee) 2 xy ( d Min{0; Wier — Bijer - sei) 
rER icS, iEl jeJ kEJ rER 
When the multipliers are set to: 
Av =O VWrER, ES, 
and 
0 WIAR,tE Sr,srER RET 
le i. Vi=j,i€S,rER, kEd, 


ji is an index of j that satisfies W,;,kr = Wi... Under those conditions 
Min{0; Wijer — Bijee — Air} = O and the objective function of the 
Lagrangian is reduced to 
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[5 ZPD Aa, ie be, 

jJEd ie! ied rER keJ 

which is the objective function to Problem YLB. Thus 
Ziz = L(0, Wir) = L(X, B). 


Using similar arguments it can be shown that Z7, < L(i, B), leading 
toZip <= L(A, 8B) O 


VI. HEURISTICS FOR CONFIGURING DISTRIBUTED COMPUTER 
SYSTEMS 


The problem of designing an optimal configuration of a distributed 
computing system belongs to the class of NP complete problems. As 
such it is unlikely that an algorithm will be developed that is capable 
of solving every instance of the problem to optimality. In lieu of this 
evidence, the only avenue that is open to a systems designer is to rely 
on heuristic procedures. Those are expected to generate good but not 
necessarily optimal solutions for the problem. In this section we 
develop such heuristics. Providing a theoretical bound on the expected 
quality of the solutions generated by those heuristics is itself a hard 
problem. Therefore, the performance of those heuristics is investigated 
in a set of computational experiments in which the heuristics are 
applied to a large number of problems and their performance is 
compared to the lower bound values that were generated by using the 
procedures developed in Section V. The sensitivity of those procedures 
to various design parameters is evaluated and can help in identifying 
the conditions under which those procedures can be expected to 
generate good solutions. 

The heuristics developed in this section are of the greedy type. They 
start from an initial solution and attempt to improve it by reassigning 
data sources to database locations or reports to report generation 
locations. The solution improvement steps are based on the following 
observations. 

Assuming that the decisions regarding the assignment of reports to 
report generation locations have been made, i.e., a vector { Y;,} of zero- 
one variables has been selected, the optimization problem, Problem 
QIP, is reduced to: 


Problem YIP 


Zyp=miny » ZjP;+d » x,¥i} 
cl 


iE! jeJ 
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subject to: 
> Xij =] LE i 


jed 
Xi < Z; ieEl,jed,, 
X;,Z;=Oorl i€Ljed, 
where 
Jy ={j| Y; =landrER,j EJ} 
dy =JI—dy 
= {rji€ S, andr eé R} 
pe Viet Do Wain 


reT; 


SON 


and where k, is the index of the location in which report r is generated 
(i.e., Vp: = 1). 

Thus, when the assignment of reports to report generation locations 
is fixed, the problem is reduced to a simple plant location problem. 
Using similar arguments, one can show that, when the assignment of 
data sources to potential database locations is given, i.e., an X;; vector 
has been selected, Problem QIP is reduced to: 


Problem XIP 
Zxip = min} ¥ ZjPj + Y ps Yijr of 
jeJ, rER jeJ 

subject to: 

» Yr = reR, 

jeJ 

Y;, = Z; rER,jéd,z, 

Y;-, Z; = 0 or 1 rE R,j Ed, 

where: 


J, = {j|X; = 1 andiel,j € J} 
Jd; =J-—d, 
U;, = U;, + by Westies 
icS, 
and where j; is the index of the location to which the data originating 
from source i has been assigned (i.e., Xj, = 1). 


This problem is also a plant location problem. 
Problems XIP and YIP are the basis for the development of the 
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following class of heuristics to the distributed system configuration 
problem. The heuristic consists of the iterative application of those 
two subproblems in order to improve an initial feasible solution to the 
problem. A detailed description of the heuristic for configuring a 
distributed computing system follows: 

Step 1—Using some selection criteria, select an initial assignment 
of reports to report generation locations, i.e., form a Y;,, vector. 

Step 2—Improving the assignment of data sources to database 
locations: 

a. Using the vector Y;, as an initial assignment of reports to 
report generation locations, solve Problem YIP. 
b. From the solution to Problem YIP form an X;; vector. 

Step 3—Improving the assignment of reports to report generation 
locations: 

a. Using the vector Xi, which was generated in Step 2b of the 
algorithm, solve Problem XIP. 
b. From the solution to Problem XIP form a Y>, vector. 

Step 4—If in a full pass through Steps 2 and 3 of the algorithm, 
there was no improvement in the objective function value, Stop. 
Otherwise, go to Step 2. j 

In each step of the algorithm an attempt is made to improve the 
assignment of data sources to database locations or the assignment of 
reports to report generation locations. The process terminates when, 
in a full pass through the algorithm, there was no further improvement 
in the solution. Various methods can be used in Step 1 of the algorithm 
for generating an initial assignment of reports to report generation 
locations. Different initial assignments could lead to different final 
solutions by the heuristic. Some of the methods that have been 
considered are listed: 

1. A single location is selected for generating all the reports. This 
could be the location that minimizes the system costs and involves the 
evaluation of up to |J] different assignments of data sources to 
database locations. Each one of those evaluations requires the solution 
of a plant location problem and might be computationally expensive. 
Another possibility is to select the starting location as the center of 
gravity in an approximate gravity model, or to pick the starting node 
at random. 

2. For each report r, r € R, the location that minimizes the process- 
ing and distribution costs for that report is identified. The union of 
those locations is selected as a starting solution. 

3. The simple procedure that was developed for generating a lower 
bound to Problem ILP provides as a byproduct a feasible assignment 
of reports to locations, or of users to database locations. Those assign- 
ments can be used as input to Step 1 of the heuristic. 
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A second set of heuristics for the problem is obtained when the role 
of the Y;, and the X; variables is reversed in the suggested heuristic. 
The correct combination of starting procedures and heuristics can be 
decided based on their empirical performance in a set of computational 
experiments. 


VII. NUMERICAL EXAMPLES AND COMPUTATIONAL EXPERIMENTS 


The methods that were developed in the previous sections provide 
a system designer with lower and upper bounds on the value of the 
optimal solution to the system configuration problem, and with a set 
of possible designs for such a system. To investigate the quality of the 
solutions that are generated by those procedures, they were pro- 
grammed and tested on a set of simulated cases. In this section we 
present an example of using those procedures, and conduct a set of 
computational experiments in which the performance of those proce- 
dures is tested over a wide range of parameter values. These examples 
demonstrate the value of the design method and highlight the cases in 
which the procedure is expected to perform very well. First we present 
the data generation method for the base case. Then, by varying 
different parameters, we demonstrate how different cost factors influ- 
ence the selection of an appropriate architecture for the system. By 
comparing the cost of the computed solution to the cost of a centralized 
solution, it is possible to identify the cases in which the distributed 
configuration clearly dominates the solution offered by a centralized 
system. 


7.1 Data generation method for the base case 


The base case attempts to simulate the flow of data and reports in 
a large organization that is distributed over a large geographic area. 
The organization consists of a set of points representing manufactur- 
ing, distribution, and customer services centers. Those points originate 
the transactions needed for the management and control activities. 
The points are grouped into regions; each region has a regional center 
responsible for managing the entities that belong to its region. The 
organization has a general headquarters that receives reports and data 
from the organizational entities and is responsible for managing the 
organization. 

The data generation method consists of the following steps. A 
rectangle of dimensions 1000 by 2500 was divided into n by m regions 
of equal dimensions. In each region, the coordinates of p points were 
generated. Those points serve as data sources and can also be used for 
placement of databases as well as report generation processes. The 
geographical center of each one of those regions was set as the regional 
center location for that region. The geographical center of the rectangle 
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Table |—Test data for the computational example 


Coordinates 
Point i Xi Y; Transactions A; Entities E; 
1 416 250 20543888 4430081 
2 811 421 27995668 4661639 
3 129 455 13852528 5601720 
4 252 66 23314600 4835491 
5 416 750 17652664 5938730 
6 129 681 20787928 5319019 
7 96 882 26454072 4510396 
8 182 515 3892134 4835392 
9 1249 250 22320524 5149213 
10 1087 182 23819312 5394341 
11 1579 276 27018756 5172949 
12 1501 323 9757924 4795006 
13 1249 750 3260475 5874032 
14 1155 922 27545052 5244782 
15 1364 847 15831062 5272542 
16 1490 600 22183256 4646214 
17 2082 250 2963550 4512458 
18 2488 716 3077690 4119027 
19 2417 100 24784832 4996060 
20 1742 239 1123881 5207785 
21 2082 750 2104029 5316656 
22 2116 694 28796560 4985719 
23 1891 989 10662968 5162839 
24 1784 849 23130012 4246616 
25 1250 500 4496444 4408979 


was set as the location in which the general headquarters resides. In 
the numerical example that follows the parameters were set to n = 2, 
m = 3, and p = 4. Points 1, 5, 9, 13, 17, and 21 are regional centers, 
while point 25 represents the general headquarters. Regional center 1 
controls and receives reports from points 1 to 4, 5 controls points 5 to 
8, and so on. 

For each point i a tuple (A;, E;) was generated, where: A; ~ U(10°; 
3 x 10’) and E; ~ U(4 x 10% 6 x 10°). A; represents the amount of 
monthly update activity that originates in data source 1, while £; 
represents the number of entities in data source i, on which data has 
to be stored in the system databases. Table I contains the coordinates 
and the (A;, E;) values for the different points. 

The set R is partitioned into R,-daily, Ro-weekly, R3-monthly, R4- 
quarterly, and R;-yearly subsets of report types. Those reports are 
generated in a frequency of 365, 52, 12, 4, and one time per year. For 
each point i a daily and a weekly report are generated (reports 1 to 50 
in Table II). A copy of the weekly report is also distributed to the 
regional center that controls point 1. For each regional center weekly, 
monthly, quarterly, and yearly reports are generated. Those are reports 
51 to 74 in Table II. Monthly, quarterly, and yearly reports are 
generated for the general headquarters. Those are reports 75, 76, and 
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Table II—List of reports, their data sources, and end user locations 


Genera- 
Report ID _ tion Fre- 
r quency 
01-25 Daily 
26-50 Weekly 
51-56 Weekly 
57-62 Monthly 
63-68 Quarterly 
69-74 Yearly 
715 Monthly 
716 Quarterly 
77 Yearly 


End User Loca- 
tions 

r— 25, RC, 

(r—51)-441 

(r—57)-44+1 

(r — 63)-44+1 

(r—69)-44+1 

25 

25 

25 


Set of Data Sources for Report r — (S,) 


(RCo6 = 1, RCs = 5, RC, = 9, RCs = 13, RCe2 = 17, RC = 21, RCso = 25) 


77 in Table II. Table II lists for each report type the set of data sources 
that provide data for that report, and the list of end user locations to 
which the report has to be distributed. 

The cost entries for the numerical example are based on the yearly 
operations of the system, and the objective is to minimize the yearly 
costs of operating it. The cost factors are setup and computer operation 


costs: 


data collection, transfer, update, and storage costs: 


P; = vw500,000 j Ed, 


V; = (E:Q:C; + AiC2 + Ai4dyC3V;)-v, 


where: 
Q; = 500 bytes 
C, = 107 $/ebyte 


C. = 10°? $/etransaction 


C; — 5 X 107° $/ebyte 


= 
V;= 
v= 


and 


120 bytes 
12 
0.0005 


d;; = max{100; 10 times the distance between points 1 and j }, 


The report generation and distribution costs are: 


tin =( 
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kED, 


n,CypArlogod, + C5rA-n-L, + Cype-logee, + dy onCs) 


where: 


A; = > A; V;/n,, 
iES, 
ee, = 0.25 ¥ Ej, 
iES, 
L, = 100, 


D, = the set of locations that receive the output of report r, 
n, = the number of times per year that report r is generated, 
®, = the amount of output generated by report r (in bytes). 
®, is given by: 


®, = A, Lir as 4e,Lo, 


ie a te re R, 


50 rER—-R, 

200 re R, 
L, =4 250 reR 
ar 300 rE R; 


500 rE R,orrE R; 


f10-* when r is a report that goes to a regional 
C4, = | center or the headquarters 
2 x 10°® otherwise 


Cs- = 3 X Cu, 
pw = 0.0003. 
Data retrieval and transfer costs are given by: 
Wiikr = [CaBir( Ai Vi/n, loge (Ai Vi/n,) + Ca-hirE loge E; 


+ (Cy81gi-Ai Vi/n, + Cisohi-H;)djx]-w 
where: 


&ir = 100, hi, = 250, s,= 0.1, sy =0.25 and w=0.0015. 


s, and sz are the appropriate compression factors between the inputs 
and outputs from the databases. 

The heuristic and the lower bounding procedures have been applied 
to the base case. The heuristic procedure was initiated with a solution 
in which all the databases were assigned to point 25 (the headquarters). 
After five iterations the heuristic stopped with a solution having a 
yearly cost of approximately vw9,090,000. Seven computers were as- 
signed to the system; all of them are used for database assignment and 
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Table III—Assignment of 
data sources to database 
locations (for base case) 


Data Sources 
Database Assigned to those 


Locations Databases 
1 1, 2, 3, 4 
5 5, 6, 7, 8 
9 9, 10, 11, 12 
13 18, 14, 15 
17 17, 18, 19, 20 
21 21, 22, 23, 24 
25 25 


> 


for report generation processes. The 9 million dollars consist of 3.5 
million dollars for computer setup and operational costs, vw1,655,200 
for data collection transfer update and storage costs, vw3,509,350 for 
report generation and distribution costs, and vw426,340 for data 
retrieval and transfer costs. Tables III and IV present the assignment 
of data sources and report generation processes to computer locations. 

The simple lower bounding procedure when applied to the base case 
has generated a lower bound value of vw8,662,100, assuring that the 
gap to optimality is at most vw422,000. The cost of a centralized 
solution in which only one computer is assigned to point 25 and all 
the databases and report generation processes are assigned to it is 
vw21,022,350, demonstrating that a distributed architecture is clearly 
preferable for the base case. 


7.2 The impact of cost changes on system configuration 


One of the important aspects of having a model for configuring 
distributed computing systems is the ability to perform sensitivity 
analysis in which the impact of changes in different parameters that 
influence such a design are investigated. Many cost factors and activity 
parameters influence the final design of a computing system. Many of 
those parameters have to be estimated long before the system is 
actually developed and implemented. Therefore, it is important to be 
able to investigate well in advance what the impact of changes will be 
in parameter values and the assumptions used to derive them. 

In this section we demonstrate the use of the model to illustrate 
how the final configuration changes with changes in parameter values. 
The method used in those investigations was to multiply a parameter 
by a factor that was varied during the experiments. A factor value of 
1 is identical to the base case that was described in the previous 
section. 

In Table V we demonstrate the impact of changes in setup and 
operational costs on system configuration. The table contains: 
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Table IV—Assignment of report generation processes 
to computer locations (for base case) 


Assigned Assigned Assigned 
Report r to Report r to Report r to 
1 1 2 1 53 9 
2 1 28 1 54 13 
3 1 29 1 55 17 
4 1 30 5 56 21 
5 5 31 5 57 i 
6 5 32 5 58 5 
7 5 33 5 59 9 
8 5 34 9 60 13 
9 9 35 9 61 17 
10 9 36 9 62 21 
11 9 37 9 63 1 
12 9 38 13 64 5 
13 13 39 13 65 9 
14 13 40 13 66 13 
15 13 41 13 67 17 
16 25 42 17 68 21 
17 17 43 17 69 1 
18 17 44 17 70 5 
19 17 45 17 71 9 
20 17 46 21 72 13 
21 21 47 21 73 17 
22 21 48 21 74 21 
23 21 49 21 75 25 
24 21 50 25 76 25 
25 25 51 1 ae 25 
26 1 52 5 


Table V—The impact of changes in setup costs (P) on system 
configuration 


System Costs (in Thousands) Computer Locations 


Report 
Data- Gener- 
Factor PZ; TVX; TUerYer UWX Yr, Total bases ation Total 


32 16000 5173.54  15346.17 2.64 36522 1 1 1 
16 8000 5173.54  15346.17 2.64 28522 1 1 1 
8 8000 3640.56  10847.64 246.36 22734 2 2 2 
4 6000 2678.10 7264.51 409.21 16351 3 3 3 
2 5000 2066.68 4886.43 358.52 12311 5 5 5 
1 3500 1655.20 3509.35 426.34 9090 7 7 7 
1/2 1750 1655.20 3509.35 426.34 7340 cd 7 7 
1/4 1125 1283.62 3509.35 483.64 6401 9 7 9 
1/8 1000 331.98 3509.35 741.65 5582 16 7 16 
1/16 961.6 147.83 3509.35 809.75 5028 18 7 18 
1/32 343.2 22.4 3509.35 828.12 4703 22 7 22 


e The total cost of the solution generated by the heuristic and the 
breakdown of the total cost into its major components 

e The total number of computer locations used by that solution, 
how many of them are used for database placement, and how 
many for report generation processes. 
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Table VI—The impact of changes in setup costs (P) on the heuristic, 
lower bound, and centralized solution values 


Lower Cost of 

Bound on Cost of Heu- Lower Cost of 

Optimal So-__ristic Solu- Bound Pol- Cost of Best Centralized 
Factor lution tion icy Heuristic Solution 
32 36517.0 36522.35 36522.35 36522.34 36522.35 
16 28517.0 28522.35 28522.35 28522.35 28522.35 
8 21939.8 22734.57 22351.82 22351.82 24522.35 
4 15939.8 16351.82 16351.82 16351.82 22522.35 
2 11950.5 12311.63 12311.64 = 12811.63 21522.35 
1 8662.1 9090.90 9090.90 9090.90 21022.35 
1/2 6912.1 7340.90 7340.90 7340.90 20772.35 
1/4 5689.1 6401.62 6425.32 6401.62 20647.35 
1/8 4776.2 5582.98 5597.27 5582.98 20584.85 
1/16 4196.7 5028.54 5042.66 5028.54 20553.60 
1/32 3868.9 4703.07 4708.95 4703.07 20537.95 





Increases in setup costs lead to solutions that are closer to a 
centralized configuration. As the cost of computing is reduced we 
observe a decrease in system costs and a move towards a distributed 
solution. In Table VI we compare, for each one of the above cases, the 
values of two heuristics, one which is identical to the heuristic de- 
scribed in the previous section. The second one uses the solution 
generated by the simple lower bounding procedure, as a starting 
solution for the heuristic. The value generated by the best of those 
two heuristics is compared to the value of a centralized solution and 
to the lower bound value. The gap between a centralized solution and 
a distributed solution increases as computing costs decrease. It is also 
interesting to note that the gap between the feasible solution and the 
lower bound is small and is nonsignificant when compared to the gap 
to the centralized solution. 

In Tables VII and VIII a similar analysis is made of the impact of 
changes in data collection, transfer, and update costs (V) on system 
configuration. We observe a trend towards a distributed architecture 
as those costs increase. The number of database locations increases 
with the increase in data collection costs. Setting the data collection 
and transfer costs to zero reduces the number of database locations 
opened in the system. However, this number is not reduced to one. 
The reason for this is the fact that report generation and distribution 
costs have not been reduced and require more than one database in 
the system in order not to increase further the report generation and 
distribution costs. In this case, too, the gap between the best heuristic 
and the lower bound is nonsignificant, assuring that the feasible 
solutions are very close to the optimal ones. 

Tables [IX and X summarize the results of changing the report 
generation and distribution costs (U) on system configuration. Here 
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Table VilI—The impact of changes in data collection/transfer and 
update costs (V) on system configuration 


System Costs (in Thousands) Computer Locations 


Report 
Data- Gener- 
Factor 2P)Z; VyX; CDUeYe UWXyY, Total bases ation Total 


32 11000 604.80 3509.35 836.95 15951 22 7 22 
16 10500 555.27 3509.35 842.35 15406 21 7 21 
8 9000 1154.71 3509.35 818.58 14482 18 7 18 
4 6000 2729.13 3509.35 733.68 12972 12 7 12 
2 3500 3310.41 3509.35 426.34 10746 7 7 7 
1 3500 1655.20 3509.35 426.34 9090 es 7 a 
1/2 3500 827.60 3509.35 426.34 8263 7 7 7 
1/4 3500 413.80 3509.35 426.34 7849 7 7 7 
1/8 3500 580.09 3763.36 507.76 8351 4 7 1 
1/16 3500 323.07 3800.57 534.65 8158 2 7 7 
1/32 3500 161.53 3800.57 534.65 7996 2 7 7 


Table Vill—The impact of changes in the data collection/transfer and 
update costs (V) on the heuristic, lower bound, and centralized 
solution values 


Lower Cost of 
Bound on Cost of Lower Cost of Cen- 
Optimal Heuristic Bound Pol- Cost of Best tralized Solu- 
Factor Solution Solution icy Heuristic tion 

32 15112.0 15951.10 15951.10 15951.1 181402.25 
16 14562.4 15406.98 15406.98 15406.98 98625.52 
8 13661.9 14482.65 14482.65 14482.65 57237.16 
4 12236.4 12972.17 12972.18 12972.17 36542.99 
2 10317.5 10746.11 10746.12 10746.11 26195.90 
1 8662.1 9090.90 9090.90 9090.90 21022.35 
1/2 7834.6 8263.30 8263.30 8263.30 18435.58 
1/4 7420.8 7849.50 7849.50 7849.50 17142.20 
1/8 7214.0 8351.22 7642.60 7642.60 16495.50 
1/16 7110.5 8158.29 7539.15 7539.15 16172.15 
1/32 7058.9 7996.75 7487.42 7487.42 16010.48 


again we observe a reduction in the number of databases and report 
generation locations as the value of U is reduced. Reducing the report 
generation and distribution costs to a negligible level still requires two 
computer locations for an efficient solution. The reason for it is the 
fact that the data collection transfer and storage costs have not been 
changed. They are still significant in the total system costs and can 
be reduced only by having more than one database location in the 
system. The gap between a distributed and centralized solution in- 
creases as the value of U is increased. Here again we observe a 
nonsignificant gap between the heuristic and the simple lower bound- 
ing procedure. 

Tables XI and XII summarize the impact of changes in interprocess 
communication costs W on system configuration. A decrease in W 
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Table IX—The impact of changes in report generation and 
distribution costs (U) on system configuration 


System Costs (in Thousands) Computer Locations 


Report 
Data- Gener- 
Factor 2P)Z; UVyXzi  TFUrYir DWWXyYi, Total bases ation Total 


32 3500 1655.20 112299.27 426.34 117880 7 7 7 
16 3500 1655.20 56149.63 426.34 61731 7 7 7 
8 3500 1655.20 28074.81 426.34 33656 7 7 i 
4 3500 1655.20 14037.40 426.34 19618 7 7 7 
2 3500 1655.20 7018.70 426.34 12600 7 7 7 
1 3500 1655.20 3009.35 426.34 9090 7 7 7 
1/2 2500 2066.68 2443.21 358.52 1368 5 5 5 
1/4 1500 2678.10 1816.12 409.21 6403 3 3 3 
1/8 1000 3640.56 1355.95 246.36 6242 2 2 2 
1/16 1000 3640.56 677.97 246.36 5564 2 2 2 
1/32 1000 3640.56 338.98 246.36 5225 2 2 2 


Table X—The impact of changes in the report generation and 
distribution costs (U) on the heuristic, lower bound, and centralized 
solution values 


Lower Cost of 
Bound On Cost of Lower Cost of 
Optimal Heuristic Bound Cost of Best Centralized 
Factor Solution Solution Policy Heuristic Solution 
32 117452.2 117880.8 117880.8 117880.8 496753.5 
16 61302.9 61731.2 61731.2 61731.2 251214.9 
8 33227.7 33656.3 33656.3 33656.3 128445.5 
4 19190.3 19618.9 19618.9 19618.9 67060.8 
2 12171.6 12600.2 12600.2 12600.2 36368.5 
1 8662.1 9090.9 9090.9 9090.9 21022.3 
1/2 6851.7 7368.4 7284.3 7284.3 13349.2 
1/4 5766.7 6403.4 6199.8 6199.8 9512.7 
1/8 5042.4 6242.8 5409.6 5409.6 7594.4 
1/16 4596.7 5564.9 5340.0 5340.0 6635.3 
1/32 4308.8 5225.9 4832.6 4832.6 6155.7 


increases the incentive to move towards a distributed solution. As W 
increases the gap between the heuristic and the centralized solution is 
decreased. However, the number of computer locations is not reduced 
to one, mainly due to the nonnegligible data collection and report 
distribution costs. When interprocess communication costs are signif- 
icant, we also observe a significant gap between the lower bound value 
and the heuristic solution value. This might be one of the cases in 
which it is worthwhile to apply the Lagrangian-based lower bounding 
procedure. 

In the last set of experiments we investigate the impact of uniform 
changes in communication costs on system configuration. The method 
by which those changes were entered into the data was to multiply the 
distance measures {d,;}, which were defined in the previous section, by 
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Table XI—The impact of data retrieval and transfer costs (W) on 
system configuration 


System Costs (in Thousands) Computer Locations 


Report 
Data- Gener- 
Factor 2DP,Z; 2UVyXy  DUerYor ZWXyYer Total bases ation ‘Total 


32 2500 5173.54 9258.75 1908.21 18840 1 5 5 
5 


16 2500 5173.54 6222.17 3032.75 16298 
8 3000 5173.54 5479.41 1694.65 15437 
4 3000 5173.54 4848.00 1281.55 14303 
2 3500 1655.20 3509.35 852.68 9517 
1 3500 1655.20 3509.35 426.34 9090 


1/2 3500 1655.20 3509.35 213.17 8877 
1/4 3500 1655.20 3509.35 106.58 8771 
1/8 3500 1655.20 3509.35 53.29 8717 
1/16 3500 1655.20 3509.35 26.64 8691 
1/32 3500 1655.20 3509.35 13.32 8677 


ee ee ee ee ee ee 
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Table XII—The impact of changes in data retrieval and transfer costs 
(W) on the heuristic, lower bound, and centralized solution values 


Lower 
Bound on Cost of Heu- Cost of Cost of Cen- 
Optimal ristic Solu- Lower Bound Cost of Best tralized Solu- 
Factor Solution tion Policy Heuristic tion 

32 8744.7 18840.51 22307.59 18840.51 21104.26 
16 8701.7 16928.46 15486.08 15486.08 21061.98 
8 8680.7 15347.61 12075.32 12075.32 21040.85 
4 8670.9 14303.10 10369.94 10369.94 21030.28 
2 8664.9 9517.25 9517.25 9517.25 21024.99 
1 8662.1 9090.90 9090.90 9090.90 21022.35 
1/2 8660.8 8877.73 8877.73 8877.73 21021.03 
1/4 8660.4 8771.14 8771.15 871.14 21020.37 
1/8 8660.0 8717.85 8717.85 8717.85 21020.04 
1/16 8659.9 8691.20 8691.21 8691.20 21019.87 
1/32 8659.6 8677.88 8677.88 8677.88 21019.79 


a factor that was changed during the experiments. A change in dis- 
tances implies similar changes in 

e Data collection transfer and storage costs 

e Interprocess communication costs 

e Report generation and distribution costs. 
The results of those experiments are summarized in Tables XIII and 
XIV. As expected, a uniform reduction in communication costs leads 
to a centralized solution, while an increase in communication costs 
leads towards a distributed solution. The gap between a centralized 
and the distributed solution increases as communication costs in- 
crease. A similar trend is observed between the lower bound and the 
heuristic solution. 
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Table XIII—The impact of uniform changes in communication 
distances (D) on system configuration 


System Costs (in Thousands) Computer Locations 


Report 
Data- Gener- 
Factor DP;Z; > Vi Xi > U;- Yur z WX; Yor Total bases ation Total 


32 11000 494.4 111923.3 26480.6 149898.9 22 7 22 
16 9000 2276.1 55967.7 =12944.1 80187.9 18 7 18 
§ 8000 2619.4 27989.9 5926.1 44535.5 16 7 16 
4 5000 4364.7 14000.9 2156.9 25522.6 10 7 10 
2 3500 =3309.3 7006.5 850.7 14666.6 7 7 7 
1 3900 = =1655.2 3509.3 426.3 9090.9 7 7 7 
1/2 2500 1033.6 2447.5 180.3 6161.5 5 5 5 
1/4 1500 = 535.8 1457.7 83.5 3577.1 3 3 3 
1/8 500 517.1 1536.5 2.6 2556.3 1 1 1 
1/16 500 7.3 24.8 2.6 534.7 1 1 1 
1/32 500 7.3 24.8 2.6 534.7 1 1 1 


Table XIV—The impact of uniform changes in communication 
distances (D) on the heuristic, lower bound, and centralized solution 


values 
Lower Cost of 
Bound on Cost of Lower Cost of 
Optimal Heuristic Bound Cost of Best Centralized 

Factor Solution Solution Policy Heuristic Solution 
32 123303.1 149898.9 150070.3 149898.9 657069.9 
16 66915.5 80187.9 80387.2 80187.9 328787.4 
8 38099.9 44535.5 44646.2 44535.5 164646.1 
4 22717.2 25522.6 25651.4 25522.6 82575.2 
2 13813.0 14666.6 14666.6 14666.6 41539.9 

1 8662.1 9090.9 9090.9 9090.9 21022.35 
1/2 5978.4 6161.5 6161.5 6161.5 10763.4 
1/4 3491.1 3577.1 3577.1 3577.1 4608.3 
1/8 2450.2 2556.3 2479.4 2479.4 2556.3 
1/16 529.8 534.7 534.7 534.7 534.7 
1/32 529.8 534.7 534.7 534.7 534.7 


Vill. POTENTIAL APPLICATIONS AND EXTENSIONS OF THE MODEL 


The optimization models for configuring distributed computer sys- 
tems, which were formulated in the previous sections, were based on 
simplifying assumptions. Those assumptions restrict the use of the 
model to a subset of the organizations in which we expect that 
distributed computing systems will be widely implemented. Some of 
those restrictive assumptions can be relaxed, and the model can be 
extended to a wider range of application areas. In this section we 
present a few of those extensions and provide examples to existing or 
planned systems that are prime candidates for using this model. Each 
of these systems has unique characteristics that are exploited in the 
models and the suggested solution methods. 
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8.1 Allowing for different transaction classes 


The model developed in Section IV has assumed that all the trans- 
actions that originate in a data source are assigned to the same 
database location. This assignment is regardless of the set of entities 
and functions that are being served by the data contained in those 
transactions. This is rather a highly restrictive assumption. In many 
of today’s organizations it is difficult to associate a single data source 
with a unique set of functions that are served by the transactions that 
originate in that data source. Situations in which the same data source, 
and sometimes the same individual, provides input data to many 
databases, and to a variety of sometimes conflicting functions, are 
quite common. For example, a salesperson in a company sales office 
can report orders for final products that were received from customers, 
which will update a database that supports the order handling subsys- 
tem. The same salesperson can also enter data on payments made by 
customers, fill out presence or absenteeism reports that will update 
the personnel and employee history database, or can enter customer 
reports on problems that they have encountered with equipment 
installed by the company (those transactions will typically update an 
engineering or technical support database). When larger entities are 
used as data sources (such as departments, branch offices, plants, or 
divisions), then the association between data sources and functions 
becomes less apparent. For example, a job shop in a large manufac- 
turing company originates and feeds transactions that deal with dif- 
ferent activities in the organization. Transactions can report on the 
progress of jobs in the production process, the consumption of inven- 
tory items in those jobs, transfer of final goods to inventory, machine 
and equipment failure, presence or absence of employees. Those trans- 
actions will be routed to different databases and will be used by 
completely different sets of end users in the organization. The events 
reported by those transactions occur at different rates, and they are 
of interest to completely different sets of end users that might be 
located in different regions of the country, where each one of those 
groups might have their own reporting requirements. 

The restriction that all the transactions that originate in a single 
location will be routed to the same database location, can, in many 
instances, increase the data processing costs to the organization that 
is being served by the data processing department and, at the same 
time, lengthen the response times to user requests for reports. Orga- 
nizations that have such characteristics can benefit from relaxing this 
restriction. It might be possible to reduce the system’s data processing 
costs and improve its response time by routing transactions that serve 
identical functions to database locations that are different from those 
that serve other functions even if the transactions originate from the 
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same data source. Relaxing this assumption slightly complicates the 
routing of transactions in the network, but is justified by the potential 
reduction in the system costs. 

Defining by F the index set of the various functions in the organi- 
zation, it is possible to partition the set of transactions that originate 
in source i, i € I, into up to | F| subsets of transactions, where each 
subset corresponds to a specific function in the organization. Letting 
i; denote the subset of transactions in source i that are used by function 
f, f € F in the organization, this problem can be formulated as follows. 

Find binary variables Z;, Xnj, Yi, that satisfy 


Zrigp = min x Z;Pj + y; y > Xmj Vmj + y; y Yur Ub 
Ee 


jJEd fer rER keJd 


FIED EY LT Awe Wain} 


rER ked jeJ fEF m&i;y 
subject to eqs. (3) through (5) and 
Y Xn =1 mei fe, 


jEéd 
Anji = Zj méyfeEF,j Ed, 


where V,,; and W,,j., are the appropriate cost factors for those data 
sources and functions. This optimization problem has the same char- 
acteristics as the single-function/multiple-sources system configura- 
tion problem. The only difference between the two problems is in the 
increase in the number of data sources introduced by the splitting of 
single data sources into multiple data sources. The same methods that 
have been developed for solving Problem ILP are applicable to the 
above optimization problem. 


8.2 Different setup costs for report generation than for data storage 


The hardware and software requirements of a node in the distributed 
system could be dependent on the activities performed in the node. 
For example, different setup costs might be incurred for supporting a 
database versus supporting a report generation process, or for sup- 
porting both activities. 

Letting P;:, t = {1, 2, 3} be the setup costs for placing in location j 
a computer that can support a: database {1}, report generation process 
{2}, or both activities {3}. We redefine the Z; variables in Problem QIP 
and Problem ILP as the sum of three variables, Z;,, t = {1, 2, 3}. Zj: 
are binary variables that specify what type of capabilities will be 
provided in location j. 

This problem has the same formulation as Problem QIP or Problem 
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ILP, with the Z; variables replaced by the sum 2, Z;, and the addition 
of the constraints 


3 
Y Zee JEd. 
i=1 


Techniques similar to the ones used for solving the previous models 
can be used to generate solutions for this problem. 


8.3 Transactions that update multiple databases 


A situation that arises in many systems is one where a transaction 
is used to update two or more functionally different databases that 
might be located in different locations. For example, a transaction 
that reports on the transfer of inventory items from one regional 
warehouse to another warehouse might have to update two fragments 
of the inventory database that are located in different locations. 
Transfer of employees from one region to another, transfer of items 
from inventory to the production process, transfer of final products 
from the production process to the distribution system, or third party 
charging of long distance phone calls are other examples in which 
there might be a need to simultaneously update more than one frag- 
ment of a database. 

This problem can be handled in a similar fashion to the one used 
for modeling different transaction classes by splitting the flow of 
transactions according to the databases that they have to update and 
introducing a new set of variables and costs associated with the 
synchronization of simultaneous updates to multiple databases. 


8.4 Allowing for multiple copies of databases 


An underlying assumption of the model was that there is only a 
single copy of each fragment of the database in the system. In some 
cases it is possible to save on retrieval costs by allowing for a partial 
or full overlap between databases. An example in which such dupli- 
cation can be useful is in a videotex system in which we can have full 
or partial overlap between databases. Models that select the fragments 
to be duplicated, decide which portion of each fragment will be dupli- 
cated, and assign each fragment to a computer location are not easy 
to formulate and will require further research. 


IX. CONCLUSIONS AND SUGGESTIONS FOR FURTHER RESEARCH 


We have developed models and algorithms for designing an archi- 
tecture of distributed computer systems. The algorithms have been 
tested on a few examples and have performed well by generating good 
initial designs for those systems. Research is now in progress on 
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extending those methods by developing better procedures for comput- 
ing lower bounds on the optimal solution and improved heuristics for 
generating feasible solutions for the problem. Despite the restrictive 
assumptions that have been used for developing the models, we believe 
that the developed procedures are valuable tools for helping the 
preliminary design of distributed computing systems. 

We have already pointed out the possible extensions of the formu- 
lations and the models. Another extension that might be worthwhile 
to pursue includes multiperiod versions of the problem. The models 
developed in this paper are single-period versions of the problem, 
which are well suited for the design of stable systems. In reality some 
systems go through transitional expansion periods for which multi- 
period versions of the problem have to be developed. The models 
presented here are also uncapacitated versions of the problem. Again, 
in reality we expect to have capacitated or capacity-dependent setup 
and operational cost versions of the problem. 
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Code generation techniques are used to program an application character- 
ized by complexity arising from many special cases, and rapid changes due to 
advances in the state of the art. A formal notation—an inverted decision table 
written in a propositional logic form—is developed as a means for allowing 
expert users to describe the application in a knowledge base that code gener- 
ators then can use to create production code. The complete system described 
in the paper automatically transforms a one thousand-page specification into 
a running program. The development of this system is an example of the 
formalization of the specification of a complex application. In this case the 
application is a part of the Job Management Operations System, an opera- 
tional support system to aid regional Bell Operating Company construction 
and engineering processes. The techniques described, however, can be gener- 
alized. 


I. INTRODUCTION 


A complex applications program that involved enumerating and 
analyzing many special cases was successfully developed using the 
notation, tools, and techniques described in this paper. Such enumer- 
ative complexity is characteristic of many applications. In such cases 
a detailed knowledge of the application is in the minds of experts who 
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are best able to state the rules of the game, but are not able to program 
them. Computer scientists, on the other hand, can program but do not 
know the application in depth. Thus, the skills of these two groups 
are complementary. Formal notation and tools enable them to coop- 
erate by allowing them to focus on their respective skills. 

In addition many applications have a knowledge base or set of rules 
that changes with advances in the state of the art. Since the most 
current information about the application is in the hands of expert 
users, it is desirable for them, and not programmers, to maintain the 
knowledge base of the program. This is possible when an appropriate 
notation exists for stating the application information, and when 
programming tools are available for exploiting this repository of infor- 
mation. 

In this paper we discuss the development of a module of the Job 
Management Operations System (JMOS) application software. This 
development made a deliberate attempt to incorporate both (1) nota- 
tion to allow expert users to describe their knowledge base in a formal 
specification, and (2) programming tools to generate the application 
code directly from this specification. Updating the programs to reflect 
changes in the state of the art then consists primarily of modifications 
to the knowledge base driving the program, and these updates can be 
performed directly by the expert users. 

In the software development method that evolved, the specification 
is developed using UNIX™ operating system shell and editing tools. 
This specification is in a form that can be used as a basis for a 
document for users, and also as a basis for mechanical translation into 
programs. From the specification, decision tables are derived by shell 
programs. Then these decision tables are translated into PL/1 pro- 
grams by a decision table processor developed for this application. If 
the specification has been properly composed, it then becomes a 
directly compilable program and is the primary program description. 
Although we have not achieved 100 percent of this goal, we feel that 
it is a realizable objective. 

In Section II of this paper we describe the application and those 
characteristics motivating the present work. In Section III, the evo- 
lution of our approach is presented as it proceeded through several 
iterations. Section IV details the current approach to the specification- 
design process. The last section is an assessment of this methodology. 

Our intended audience is programmers, specifiers (analysts and 
systems engineers), and managers involved in applications develop- 
ment. 


1.1 Related work 
Decision tables are well known and have an extensive literature. 
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Their use in specifications as a methodology relevant to correctness is 
described in Gerhart and Goodenough.’ Our inverted decision tables 
resemble production systems.” Computer contains a survey of recent 
work in specification languages.* Our own efforts in this area are 
somewhat more special purpose. The emphasis on productivity of 
programmers is part of the broader concern for productivity gains in 
the general economy. Jones is a collection of articles describing work 
in this area.” 

Code generation, long a part of compiler technology,® rises to a 
higher level in application code generators. Reference 7 is a broad 
survey of the state of the art in 1977, and specifically discusses decision 
tables. The work of one of the authors in an earlier application has 
been reported in Levinson, Levy, and Salisbury.® Other recent work is 
described in Refs. 9 through 14. 


Il. PROBLEM DESCRIPTION 
2.1 The application 


The JMOS system is a predominantly on-line operations support 
system that tracks and manages construction work in the outside 
telephone plant.* Its major functions are to 

e Analyze data from engineering drawings and compute the work 
content of the construction job; 

e Schedule such jobs to meet due dates, taking into account material 
availability, job priorities, and construction resources; 

e Use daily reports of work progress to track the status of jobs; 

e Analyze and report the ongoing performance of the construction 
forces; 

e Interface with accounting, budgeting, and inventory systems to 
maintain company records, produce payrolls, and track operations 
and costs. 

The application addressed in this paper deals with the first of the 
functions listed above. In particular it decomposes each work operation 
specified on an engineering drawing into two sets of components. One 
set comprises the physical tasks that must be undertaken to complete 
the work operation. The second set comprises theoretical elements 
that are used to compute a standard company work measurement 
index. 

Associated with each of the tasks in the former set is a Standard 
Time Increment (STI) allotted to perform the task. These STIs are 


* The outside telephone plant is the physical part of the telecommunications network 
that extends from the local central office to the customer’s premises. It comprises cables, 
service wires, interconnection facilities, signal regeneration equipment, etc., as well as 
supporting structures such as poles, guys, anchors, strand, manholes, and conduit. 
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later accumulated and used as the basis for planning, scheduling, and 
tracking the construction jobs. Associated with each of the elements 
in the latter set is a quantity of Work Units (WUs), which are a 
relative expression of the value and complexity of the element. This 
measure allows construction work to be compared to dissimilar work 
in other areas of the telephone business. For example, craft perform- 
ance is measured in terms of WUs per hour and cost performance is 
measured as dollars per WU. 

To identify the STI components, the application program must 
derive and correlate information about such things as the nature of 
the work to be performed (e.g., install a cable), the location (e.g., 
buried in a trench), the field conditions (e.g., rocky soil), and the 
equipment to be used (e.g., backhoe). Derivation of the WU elements 
is similar in concept, but differs significantly in terms of analytic 
detail. Throughout this paper we will use STIs as illustrations, since 
they are generally the more complex of the two work components. 

As an example, Fig. 1 is an annotated engineering drawing that calls 
for the installation of a new buried cable from a manhole to a pedestal 
(splice point). Note that there are four construction work operations, 
called job steps, in this example. Step 1 involves placing 500 feet of 
cable, 100 feet in a duct extending from the manhole, and the remain- 
ing 400 feet in a trench. Step 2 involves the placement of a terminal, 
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Fig. 1—Sample engineering drawing. 
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STEP ST! CODE DESCRIPTION QUANTITY TIME 


1 5P01 SITE SETUP - UNDERGROUND PLACING 1 0.9 
5P14 PUMP MANHOLE 1 0.3 

5P10 PLACE CABLE IN LATERAL DUCT 1 0.8 

4P01B SITE SETUP — BURIED CABLE PLACING 1 0.4 

4P08 DIG TRENCH WITH TRENCHING MACHINE 400 1.9 

4P09 ADD FOR ROCKY SOIL 400 1.0 

4P13 PLACE CABLE IN TRENCH 400 0.6 

4P15 BACKFILL TRENCH BY MACHINE 400 10 

TOTAL 6.9 

2 4P17 PLACE PEDESTAL 1 0.5 
TOTAL 0.5 

3 so9 SITE SETUP — UNDERGROUND SPLICE 1 0.7 
S10 PUMP MANHOLE 1 0.3 

$11 SET MANHOLE PLATFORM 1 0.2 

S13A PREPARE CABLE ENDS 3 0.9 

$13C CLEAN FILLED PAIRS 600 0.6 

$13D PREPARE CABLE ENDS 3600 2.2 

S22A CLOSE Y SPLICE — NEW CABLE 1 0.8 

$39 JOIN PULP PAIRS 3000 4.8 

S41 JOIN FILLED PIC PAIRS 600 1.3 

S54 IDENTIFY AND TAG PAIRS 600 4.8 

$55 PROVIDE TONE FOR TAGGING 600 4.8 

TOTAL 21.4 

4 S06 SITE SETUP — BURIED SPLICE 1 0.4 
S13A PREPARE CABLE ENDS 3 0.9 

$13C CLEAN FILLED PAIRS 1200 1.2 

$13D PREPARE CABLE CORE 1225 0.7 

S40 JOIN AIRCORE PIC PAIRS 25 0.1 

S41 JOIN FILLED PIC PAIRS 1175 2.6 

S60 PLACE GRAVEL IN PEDESTAL 1 0.1 

TOTAL 6.0 


Fig. 2—STI components derived for sample job. 


or pedestal, in which the cable will be spliced to other cables and to a 
termination block, from which service wire can be extended into a 
customer’s premise. Step 3 involves splicing (i.e., connecting) one end 
of the cable in the manhole to two other new cables installed on 
another portion of the drawing. Step 4 involves splicing the other end 
of the cable to two smaller cables and to a distribution terminal. 

The STI components of these steps are listed in Fig. 2. Figure 3 
illustrates the information supplied by the user, typically a clerk in 
the engineering office. The function of the application program is to 
analyze these input data and derive the correct set of work tasks. 


2.2 Problem complexity 


The complexity of the problem arises from a combination of static 
and dynamic factors. The static complexity is due to the inherent 
nature of telephone plant construction. The dynamic complexity of 
the problem is due to rather rapid changes in construction methods. 
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STEP DATA ITEM NAME ITEM 1 ITEM 2 ITEM 3 ITEM 4 


1 WORK ENVIRONMENT B 
MATERIAL DESCRIPTION AJTW-6 
PLANT ACCOUNT CODE 45C 
MATERIAL QUANTITY 500 
PLACING ACTION LDUC STAN 
ACTION QUANTITY 100 400 
2 WORK ENVIRONMENT B 
MATERIAL DESCRIPTION FCT-25 
PLANT ACCOUNT CODE 45C 
MATERIAL QUANTITY 1 
PLACING ACTION STAN 
3 WORK ENVIRONMENT U 
PLANT ACCOUNT CODE 5C 
FIXTURE FLAG N 
CABLE DESCRIPTION CSDC-18 AJTW-6 CSDC-12 
CABLE STATUS N N N 
CABLE CATEGORY c F F 
CABLE IDENTITY 05 05 05 
PAIR RANGE 1-1800 1201-1800 1-1200 
4 WORK ENVIRONMENT B 
PLANT ACCOUNT CODE 45C 
FIXTURE FLAG Y 
CABLE DESCRIPTION AJTW-6 AJTW-3 ASTW-3 FCT-25 
CABLE STATUS N N N N 
CABLE CATEGORY Cc F F F 
CABLE IDENTITY (1) 05 05 05 
PAIR RANGE (1) 1201-1800 1201-1500 1501-1775 1776-1800 
CABLE IDENTITY (2) A 
PAIR RANGE (2) 1-25 


Fig. 3—Input data for sample job. 


2.2.1 Static factors 


Telephone plant construction involves the use of diverse materials, 
methods, field conditions, equipment, and types of operations. In 
particular this work employs over five thousand different material 
items in over 50 classifications. Some 265 different work tasks and 80 
work-unit elements characterize just two major classes of work oper- 
ations—placing and splicing. 

Placing work includes the installation, removal, and rearrangement 
of materials and entails the use of many different types of specialized 
equipment and methods. There are also four distinct field environ- 
ments to consider: 

1. Aerial—cable and other materials supported by poles; 

2. Buried—cable buried directly in the ground; 

3. Building—cable and other materials placed on and within build- 
ings; and 

4. Underground—cable installed in underground conduits extend- 
ing between manholes. 

The following illustrates the complexity just of the installation of 
buried cable: 

There are two common installation methods—plowing and trench- 
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ing. With plowing, one of two types of vehicles may be used—one with 
a static plowshare or one with a vibrating plowshare. With the trench- 
ing method, either a backhoe or a specialized trenching machine may 
be used to dig the trench. In some cases trenches are even dug by 
hand. For each of these methods and types of equipment the system 
must assign work tasks and associated times. Additional time is 
required if the soil is rocky. Furthermore, there are some ten special- 
ized operations, used in conjunction with or in place of these standard 
methods, that must be selected on a case by case basis for each job 
step. 

Splicing is the second major class of work performed by the con- 
struction forces. Here there are fewer methods and types of equipment 
involved than with placing. However, the normal splicing process is 
more detailed and complex than its placing counterpart, so that a 
given step often involves many more work tasks. Consequently, the 
analysis of splicing input is inherently more difficult and requires the 
use of rather esoteric algorithms. 

For example, consider Step 4 in Fig. 1. As Fig. 1 shows, the input 
for this splice includes a description of each cable, along with the 
identification of its component pairs. Consider the cable at the top 
right side of the splice. It is designated as AJTW-3, which means it 
contains 300 pairs, has color-coded plastic insulation on the copper 
wires (known as PIC cable), and is filled with a petroleum jelly 
compound to render it waterproof so that it can be buried directly in 
the ground. The pairs in the cable are numbered 05, 1201 through 05, 
and 1500. Algorithms in the splicing module examine the pairs in each 
cable and determine which are being connected, disconnected, or 
rearranged. Subtotals, computed by type of cable (e.g., filled PIC), are 
used to derive the quantities of STIs and WUs for the splice. For this 
splice a total of 1175 filled PIC pairs and 25 regular PIC pairs are 
being joined. Similar analyses are performed to derive the other STIs 
and WUs for the step. 

Describing the work operations outlined above requires the use of 
five different data-entry formats. The original word specification 
describing the application was over 150 pages long. The new specifi- 
cation described in this paper is over one thousand pages long. 


2.2.2 Dynamic factors 


The application faces additional complexity due to dynamically 
changing construction operations. New materials, tools, and tech- 
niques are constantly emerging. The application must be able to be 
modified frequently to keep abreast of these changes. For example, 
lightguide cable is currently being introduced very rapidly. This re- 
quires new placing and splicing procedures. 
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In addition to this normal evolution, the problem was recently 
complicated further by the reissuance of completely revised sets of 
both work tasks and work-unit elements. These revisions were trig- 
gered by the recent completion of extensive statistical studies of 
telephone construction methods. 


Hil, EVOLUTION OF APPROACH 
3.1 Informal word specification 


Development of the prototype version of JMOS was started in late 
1980. At that time the preliminary specifications described the deri- 
vation of work tasks (and associated standard times) to be associated 
with different outside plant construction operations. The document 
was over 150 pages long and its interpretation required extensive 
discussions between the software developers and the system engineer. 

An example of the style of the original specification is the following: 


Original Specification Style 


If the step is entered on a Placing Work mask, the work environment is “buried,” the 

material class is “cable,” and one of the following conditions is met: 

e The placing action is “trench with backhoe,” “trench by machine,” or “trench by 
hand.” 

e The placing action is “standard,” the account code classification is “c,” and the 
Profile Table specifies trenching with a backhoe or a trenching machine as the 
standard buried cable placing method. 

Then for each foot of cable, generate: 


14P13 Ple Ca in Trench 


Although this example captures the flavor of the original specifica- 
tion, it does not indicate the level of complexity of the specification or 
of the changes that resulted from either errors in specification or 
typing. Suffice it to say that there were pages of the original specifi- 
cation in which the marginal notes and corrections were comparable 
to the typing on the page. Indeed, the changes were so extensive, and 
the specification so complex, that it was never reissued with correc- 
tions. 


3.2 Manual translation to design 


At that time it seemed clear that the amount of detail in the 
specification document, and the need for interpretation, would prob- 
ably induce a fairly extensive debugging effort once the programs were 
written. When coupled with the fact that there were also undoubtedly 
errors in the specification itself, that some parts of the specification 
were incomplete, and, finally, that the specification would be exten- 
sively revised in 1982, we decided to use decision tables. 
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3.3 Decision table and processor 


Decision tables are a well-known schema for specifying algorithms. 
They are, in effect, a deterministic version of the production systems 
currently popular in artificial intelligence applications. The advan- 
tages of decision tables are that they are quite change tolerant and 
fairly readable. Their disadvantage is that they represent a natural 
solution to only a part of the problem—there are algorithms that do 
not readily fit into decision table form. 

A search of various software resources failed to turn up any decision 
table processor, both available and supported, that could generate 
PL/1 code—although we did find several that met two of these three 
criteria. Accordingly, a decision table processor was built that has 
been used in JMOS for the development of an operational prototype. 

The format of the decision table that was used is shown below: 


ID-TABLE:sti-p-bu 
VERSION: %1%, DATE, %D% 


%logical 

buriedpla 

cable 

backhoe 

trencher 

hand 

standard 

pacc 

opf_backhoe 

opf_trencher 

gconditions 

buriedpla 
masktype=’p’ & workenv= ’b’ 

cable ; 
mattype = ’cable’ 

backhoe 
plaatn=/’ctrb 

trencher 
plaatn=/’/ctrm 

hand 
plaatn=’ctrh’ 

standard 
plaatn=’stan’ 


v 


’ 


Ppacc 
paccls='c’ 
opf_backhoe 


opf_tbl(’bu_cbl_mth’ ) = ’backhoe’ 
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opf_trencher 
opf_tbl(’bu_cbl_mth’) = ’trencher’ 
factions 
s4p13 
CALL add_sti(’4p13’ ,matqty, jobstep_ptr, index, 
total); 
$indexer 
%specifications 
buriedpla & cable & (backhoe | trencher | hand | 
(standard & pacc & (opf_backhoe | opf_trencher) ) ) 
%4p13 


The interpretation of this decision table specification is as follows: 

e Lines beginning with ’!’ are comment lines and are ignored by 

the decision table compiler. 

e Lines beginning with a ’%’ are syntactic markers, which denote 

the headings of sections of the decision table. 

There are four sections to the decision table. 

1. Logical—This section consists solely of a list of the logical 
conditions occurring in the decision table. It is used to compile decla- 
rations in the object program. 

2. Conditions—This section assigns values to the logical variables. 
In the object program, buriedpla will be true if masktype is ’p’ and 
workenv is ’b’. The decision table will compile this into an assign- 
ment statement initializing the logical variable. 

3. Actions—This section identifies action stubs of the decision 
table. In general, the actions are a sequence of PL/1 statements and 
PL/1 preprocessor statements. In the example, s4p13 consists of a 
procedure call, and a preprocessor statement. 

4, Specifications—This section relates conditions and actions. A 
set of specific actions to the right of the ‘%’ symbol will be invoked if 
the Boolean expression to the left of the ‘%’ symbol is true. 

The decision table processor is a very straightforward implementa- 
tion in the UNIX system, written in a combination of shell-level tools, 
primarily awk. Although there was no optimization performed in 
generating object code from the decision table, the performance was 
quite acceptable supporting our thesis that, in general, application 
code is not performance limited. However, had performance been a 
problem, it would have been possible to include optimization in the 
decision table processor without changing the decision table source. 
(The source code of the decision table is almost two orders of magni- 
tude larger that the source code of the decision table processor, so 
clearly the processor would have been the place to optimize.) 

The decision table approach did, in fact, accomplish what we thought 
it would. It was change tolerant, and more readable if only because it 
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was more concise than the PL/1 code. The decision table source 
resides in some 17 decision tables, each of which translates into a 
PL/1 program. These 17 PL/1 programs comprise in excess of ten 
thousand lines of code. 

But once extensive debugging was under way, the decision table did 
not satisfy our needs completely, since there was no direct way to 
relate outputs to inputs. The same work task output could be generated 
by any one of a number of decision tables since these were organized 
by generic categories of the application. Thus, the problem was to 
develop a means of tracing an output back to its input. 


3.4 Decision table inverter 


The format that evolved to solve this problem of correlating outputs 
to inputs was an inverted decision table. These tables, which are 
generated by shell programs, list each output, the programs that 
generate each output, and, for each program, the conditions under 
which the output arises. This inverted decision table, which is an 
extended form of cross-reference listing, made the debugging easier 
and became an invaluable aid in this phase of the project. 

The concept of a decision table inverter can best be visualized by 
considering the tabular form of a decision table: 






Conditions 






Actions 


In this standard form of decision table, the relationship between 
conditions and actions is specified by the vertical columns. Each 
vertical column corresponds to a given set of input conditions. To 
determine what output actions apply, one starts with the specification 
of input conditions and searches for a column that matches those 
input conditions. The corresponding action portion of that column 
defines the output actions in that case. 

Suppose that one wanted to know what input conditions could 
generate a given output. Then, even though the information is con- 
tained in the decision table, it is not in a convenient form. On the 
other hand, if one inverts the decision table, then the actions appear 
as the entry points that one uses to search for conditions that could 
generate those actions. The inverted decision table for our example is 
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Actions 


Conditions 





C1 car 
C2 T 
C3 T 


Although the inverted decision table is the current conceptual model 
of our methodology, it was first conceived of as a generalized cross- 
reference listing to assist in debugging the prototype software. This 
generalized cross-reference listing, or inverted decision table, was 
produced automatically from the decision table source listings. 

The major problem still remaining was that the translation from 
the specifications to decision tables was very labor intensive, entailing 
considerable human interaction between programmers and systems 
engineers to interpret the specifications. To solve this problem, the 
inverted decision table was used as a model for the next iteration of 
the specification documents. In this next iteration, the specification is 
used as an input to a processor that directly generates the decision 
tables, as an intermediate step, and then the PL/1 code. The specifi- 
cation itself then acts as a cross reference and is accurately embodied 
in the code. 

It would, of course, be possible to produce the object code directly 
from the formal specification without producing the intermediate form 
of decision table. However, there were at least three reasons for not 
doing this: 

1. In generating software it is desirable to decompose the process 
into a sequence of simple transformations such that each transfor- 
mation is easy to implement efficiently and correctly. 

2. The intermediate form of the code represented by the decision 
table can be used to provide several different versions of the final 
program. Some of these versions may have more extensive diagnostics 
to be used in the debugging phase. In fact, some of the versions were 
so used. 

3. We had at this stage of the project a working decision table 
processor that did a significant part of the transformation for us. 


IV. CURRENT APPROACH 


The development of the prototype system began in the fall of 1980, 
and at that time the decision table processor was written. During the 
first half of 1981, the decision table processor was used to produce 
PL/1 code. In the late summer of 1981, the decision table inverter was 
written to assist in the debugging of the prototype. In planning for the 
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production release, we decided to use the inverted decision tables as a 
model for the software specification itself. 

The current approach requires that a systems engineer write a set 
of specifications, which are then automatically translated into produc- 
tion software. The specification must conform to a rigid documenta- 
tion format and use a propositional-calculus-type language with a 
limited vocabulary and a simple syntax. The specification is machine 
readable so that the two-stage code generation program can transform 
it first into decision table format and finally into PL/1 code. 

Section 4.1 details the specification generation process. This in- 
cludes a description of the specification structure and the language 
developed for the application. Also described are several software tools 
designed to aid in the preparation of the specifications themselves. 

Section 4.2 describes the process of translating the specifications 
into finished code. This includes the two stages of the translation 
process as well as the handling of several special problems, such as 
the addition of program elements not expressly included in the speci- 
fication. 

Section 4.3 discusses the performance improvements that result 
from this current approach. These include programming efficiency, 
streamlined debugging, and vastly simplified maintenance. 


4.1 Specification: language and preparation 


4.1.1 Document structure 


The specification document contains three parts—logic modules, 
data structures, and function definitions. A logic module defines a set 
of conditions required to generate a single STI task or WU element. 
There are about 650 of these modules, which constitute over 90 percent 
of the total specification document. The data structures define the 
vocabulary of the specification language. There are five parts to the 
data structure section, corresponding to the five data-entry formats. 
The function definition sections define 20 special functions referenced 
in the logic modules. These functions perform numerical computations 
or evaluate logical conditions that either cannot be handled by the 
logic modules or that are common to many logic modules. 

Presently our code generation tools work only on the logic modules. 
The data structures are used to manually produce the PL/1 data 
declarations. The 20 functions are manually coded. 

4.1.1.1 Logic modules. Each logic module comprises six parts (see 
Fig. 4, which defines the conditions for STI task 4P 13). 

1. Title—The first line of the module is the title. It indicates 
whether the module applies to an STI task or to a WU element and 
identifies the specific task or element involved. Thus if the conditions 
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Description: 
Place cable in a trench. 


Level: job step 
Factor: matqty 
Conditions: 
Mmasktype = epw & 
<like_cable> = “true” & 
{ plaatn = ctrb 
Plaatn = ctrm 
Plaatn = ctrh 
plaatn = ctr l 
wrkenv = b & 
plaatn = stan & 
pacclass = c & 
OPF(bu_cbl plc _mth,argl) “= plow 
} 
} 
Notes: 


1. No work environment check against actions ctrm, 
ctrb, ctrh, ctr, since these apply 
unambiguously to buried plant. 


2. Actions bore and push are not allowed here 
(i.e., to generate cable placement), since they imply 
only that structure work is to be done. If cable is 
placed in conjunction with this other work, then the 
EPW mask should be used to enter multiple actions. 


Fig. 4—Logic module for STI 4P 13. 


specified in the remainder of the module are satisfied, then some 
quantity of this task is required to work the job step being analyzed. 

2. Description—The second section is a word description of the STI 
task or WU element denoted by the title. In cases where more than 
one logic module applies to a given task, the description differentiates 
among the cases. The purpose of the word description is simply to aid 
persons reading the specification. 

3. Level—The third section is a one-line entry that specifies the 
operating level of the logic module. The set of possible levels that may 
apply are defined by the data structures document and correspond to 
the hierarchical layers of data defined for each of the five entry forms. 
For example, for a placing step entry form, the possible levels are job 
step, supplementary placing action, and plant account code. 

The level factor controls the processing of the logic module. The 
conditions in the module are evaluated once for each appearance in a 
job step of the data denoted by the level parameter. Thus if the level 
is job step, the module is evaluated once for a step. In the placing step 
example cited above (see also Fig. 4), the level is supplementary placing 
action, so the module is evaluated once for each action entered for the 
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step. (From 0 to 20 of these entries are allowed for a single placing 
step.) 

The reason for varying levels is that different tasks apply at each of 
these levels. For example, since the task of setting up a work site 
applies to a job step as a whole, the logic module for a site set-up task 
is evaluated only once (job-step level) per step. On the other hand, a 
task associated with a placing action (e.g., STI 413 as shown in Fig. 
4) must be evaluated for each action entered for a step in order to 
determine how often it applies. 

4, Factor—The fourth section is another one-line entry. It specifies 
the quantity of the associated task to be generated if the conditions of 
the module are satisfied. This value must be numerical. It may be a 
fixed number (a constant), a variable name corresponding to an input- 
data item, or the result of a function computation. In the example in 
Fig. 4, the factor is the quantity associated with the placing action 
and represents the length in feet of the trench to be dug. 

5. Conditions—The fifth section is the main body of the logic 
module. It defines the logical conditions that must be satisfied in order 
to generate the associated task. The conditions are written in a 
propositional-calculus-language form developed specifically for this 
application (see Section 4.1.1.2 for a detailed description). 

The conditions specify the various combinations of data values that 
indicate the need for a given STI task or WU element. They may 
include references to data entered by the user, parametric data derived 
from these entries, the output of functions applied to these entries, or 
fixed parametric data extracted from tables that define the standard 
operating methods used by the local construction organization. 

6. Notes—The last section contains an optional set of notes. Notes 
are provided to explain certain aspects of the conditions that may not 
be obvious to the reader. They may also be used to further differentiate 
a given module from others that apply to the same STI or WU.* 

4.1.1.2 Data structures. Each data structure lists the set of data items 
used in the logic modules and functions associated with a particular 
data-entry form. Figure 5 illustrates the data structure for the placing- 
step entry form. 

Note that the data items are partitioned by operating level (see 
Section 4.1.1.1). For placing steps there are three such levels. The job- 
step level contains all those data items that apply universally to the 
step (e.g., only one material description—matdsc—is entered for a 
placing step). The second level is supplementary placing action. Two 


* Typical reasons for specifying multiple logic modules for a given task are that the 
task can be generated from more than one input form, at more than one data level, or 
with more than one quantity factor. 
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Structure: EPW 


Description: 
Data structure for EPW (placing work) job steps. 


Layers: step{l] supp_plc_action[2] plt_acct_code[2] 


Contents: 


STEP 

jobstp# -- job step number 

rskeystp -- RS key step flag 

rsgrpid -- RS group i.d. 

masktype -- mask type 

pacclass -- stppac(l) class 

wrkenv -- work environment 

matdsc -- material description 

matclass -- material class 

mattype -- material type 

matsize -- material size 

Plaby -- placed by indicator 

factstub -- factory stub flag 

matqty -- material quantity 

plaatn -- placing action 

rdsd -- road side flag 

hivitpro -- high voltage protection flag 
#suppatn -- number of supplementary placing actions 
#stppac -- mumber of plant account codes 


SUPPLEMENTARY PLACING ACTION 


suppatn -- supplementary placing action 
atngqty -- supplementary placing action quantity 


PLANT ACCOUNT CODE 


stppac -- plant account code 

pacqty -- plant account code quantity 
paccls -- plant account code class 
meas -- measured account flag 
plittype -- plant type 


Notes: 


1. Note that the SUPPLEMENTARY PLACING ACTION & PLANT ACCOUNT CODE 
portions of the data structure are parallel. They each relate 
directly to the STEP level. 


Fig. 5—Data structure for placing data-entry forms. 


data items are associated with each supplementary placing action in a 
step. Up to 20 pairs of such data may be entered for a placing step. 
The third level is plant account code. Two data items are also associ- 
ated with each plant account code and up to three pairs of such data 
are permitted per step. Note that the second and third levels in this 
case are parallel, in that each homes on the job-step level. For other 
types of steps, the data-structure hierarchy may be three levels deep. 
The location of a given data item in the data structure denotes its 
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availability to the logic modules. The general rule is that a data item 
may be used in a logic module or function for which the specified level 
is the same as or lower than the level at which the data item is defined. 
For example matdsc may be used in any placing-step logic module or 
function, since it is defined at the highest level of the data structure. 
However, suppatn may only be used in those logic modules or func- 
tions that apply to the supplementary-placing-action level. 

The data items listed in the data structures define most of the 
variables used by the specification language. These data items may be 
entered directly by the user via an entry form (e.g., matdsc), or they 
may be derived from an entered value (e.g., matclass is a parametric 
data element associated with matdsc and extracted from a table of 
material descriptions). Other than items in the data structures, the 
only data variables allowed in the logic modules or functions are 
function references (e.g., (like_cable) in Fig. 4) or references to the 
Operations Profile Table, which defines the standard operating meth- 
ods used by each local construction organization (e.g., OPF (bu_cb1_ 
plc_mth) in Fig. 4 defines the standard method for placing buried 
cable). 


4.1.1.3 Function definitions. A function is used either to perform 
numerical computations, which cannot be handled by the propositional 
calculus language, or to evaluate complicated logical conditions that 
appear in numerous logic modules or functions. The function defini- 
tions are structured much like the logic modules. Each has seven parts 
(see Fig. 6, which defines the logical function (1ike_cable) and Fig. 
7, which defines the numerical function (#reg_pr_trans_sp)). 

1. Title—Name of the function. 

2. Description— Word description of the function. 


Function: <like_cable> 

Description: 
Determine whether material item for step is cable 
or some item similar to cable (eg, ground wire, 
air pipe, innerduct, or lightguide). 
Used on EPW steps and with RS logic. 


Level: job step 
Step Type: epw 
Data Type: logical 


Conditions: 
Matclass = cable | 
mattype = ground wire 
Mattype = electrolys wire | 
mattype = air_pipe 
{ matclass = fiber_op_eqp & 
{ mattype = innerduct | 
mattype = lightguide 


Fig. 6—Definition for logical function (like_cable). 
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Function: <#reg_pr_trans_sp> 

Description: 
Calculates the total number of regular pairs transferred for 
an SP step. 


Level: job step 
Step Type: sp 
Data Type: numeric 
Algorithm: 
total = 0 ; 
for ( i= 1; i <= #cblsht; i++ ) 
total = total + #trspr ; 


return ( total - #spcpr ) ; 


Fig. 7—Definition for numeric function ( #reg_pr_trans_sp). 


3. Level—Operating level of the function. It is basically the same 
as defined for the logic modules (see Section 4.1.1.1). There is one 
slight difference, however: In numerical functions it is possible to 
selectively operate at levels lower than the one specified for the 
function as a whole by means of an iteration segment. These segments 
are denoted as for loops in the function algorithm. 

4, Step type—Denotes the type(s) of data-entry form(s) to which 
the function applies. 

5. Data type—Denotes whether the function computes and returns 
a numerical value, or evaluates a set of logical conditions and returns 
a true or false value. 

6. Conditions/algorithm—The main body of the function. If the 
function is logical, this section contains the conditions that must be 
satisfied in order to return a value of true (if the conditions are not 
satisfied, false is returned). The conditions are written in the same 
propositional-calculus-language form used in the logic modules (see 
Section 4.1.1.1). If the function is numerical, this section contains the 
algorithm used to compute the numerical value. The algorithm is 
written in a predicate calculus form and includes iteration capabilities 
(patterned after the C programming language). 

7. Notes—An optional set of notes to provide an expanded expla- 
nation of the function to the reader. 


4.1.2 Specification language 


As noted previously, the conditions section of a logic module or 
function is written in a propositional-calculus-language form. This 
section describes some of the grammatical rules of that language, 
incuding the syntax and vocabulary. 

4.1.2.1 Syntax. The primary syntactic rules are: 

1. A condition statement comprises one or more logic expressions 
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<expression> t2= <exprl> | <exprl> or <expression> 


<exprl> = <expr2> | <expr2> & <exprl> 

<expr2> = <variable name><logical operator><data value> | 
{ <expression> } 

<variable name> 2 any variable listed in the data structure | 
<function> | <opf ref> 

<opf ref> ::= OPF(<opf op name><opf arg position>) 


<opf op name> 
<opf arg pos> 
<function> 
<function name> 


any operation listed in the OPF table 
argl | arg2 arg3 
< <function name> > 
any function listed in the function definitions 


wou 


<logical operator> : <lexical op> | <numerical op> 
<lexical op> = = Ts 

<numerical op> ::= = “= [< [<= [> >= 

<data value> = <variable name> <constant> 
<constant> = <number> | <string> “true’ | ‘false* 
<number> ::= any integer or decimal number 

<string> = any character string 


Fig. 8—BNF syntax of specification. 


joined by the connectors “and” (&) or “or” (|), and possibly grouped 
by curly brackets, { }. 

2. A logic expression is a comparison between a data variable and a 
data value of the form 


(variable name) (logical operator) (data value) 


3. Permissible logical operators include = and ~=, which may be 
used with any variable, regardless of type. The other operators, (, (=, 
), and )=, may be used only with numerical variables. 

4, By convention, each logic expression is stated on a separate line. 

5. By convention, curly bracket groups are indented to show the 
level of imbedding. Also, the closing curly bracket is positioned on a 
separate line directly below the opening bracket in a given group. 

6. By convention, connectors (&, or)* are placed at the end of the 
last line containing the first of the two expressions being joined. 

Figure 8 formally specifies the syntactic rules of the language. The 
Backus-Naur Form (BNF) definition given here leaves out the format 
of the specification that was incorporated as essentially a syntactic 
component. It would be possible, by extending the BNF notation to 
include horizontal and vertical positioning symbols, to include the 
format as part of the grammar. For example, if cr is used as the 
symbol for a carriage return, then the second part of the first definition 
would read (expr 1) or CR (expression). A full discussion of the 
considerations of including formats as part of the grammar would be 
tangential to the main concerns of this paper. 


4.1.2.2 Vocabulary. Other than the connectors and logic operators 
noted in the previous section, the specification language vocabulary is 
limited to two classes of terms: variables and constants. 


* In the actual object language, “or” is denoted by |. Here we have reserved the use 
of | for the metalanguage disjunction. 
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The variables have already been defined (see Sections 4.1.1.1 and 
4.1.1.2). They include the items listed in the data structures, the 
functions, and the references to the Operations Profile Table. Varia- 
bles may appear on either side of the logical operator in a logic 
expression, though they usually appear on the left. 

Constants may be numbers or character strings and may appear 
only on the right side of a logic expression. Common examples are 


plaatn=plac 
matsize <= 400 


A constant must match in type and value one of the acceptable 
values of the variable to which it is being compared. 

Two special constants are true and false. These correspond to the 
binary values of logical variables. They are used to make the specifi- 
cation more readable. 


4.1.3 Derivation 


Several programming tools were developed to aid in preparing the 
specifications. These enable the systems engineer to write a source file 
using severely abbreviated terminology and a very simple, loose format. 
The tools translate and expand the abbreviations into finished vocab- 
ulary and syntax. They also produce a finished format that adheres to 
all of the necessary conventions. For example, Fig. 9 is the source file 
corresponding to the logic module shown in Fig. 4. 

Because of the magnitude of the specification, this tool provides 
several major benefits. 

e It reduces typing significantly and enables much of the specifica- 

tion to be drafted directly on-line from working notes; 

e It eliminates the need for word-processing support to convert the 
draft into a finished document—a finished specification is pro- 
duced within minutes; 

e It reduces typing errors; 

e It guarantees that formatting is consistent and adheres to all 
conventions; and 

e It simplifies correction and maintenance. 

In addition to the translating and formatting programs, the UNIX 
operating system screen-editing tools were used to create templates of 
recurring logical conditions. These templates were then inserted where 
appropriate, usually requiring only minor editing changes or no 
changes at all. This eliminated the need to retype these sections. 
Nearly half of the specifications for placing steps were written using 
such templates. 
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gn GENERIC 1.0 
tl SPECIFICATION FOR STI LOGIC -- BURIED PLANT PLACING WORK 
id task STI 4P 


hd 13 

dsc ‘ 

Place cable in a trench. 
enddsc 

lv step matqty 

en 

epw & 

<like_cable> = ‘true” & 
{ pa ctrb | 

pa ctrm 

pa ctrh 

pa ctr | 

{ wb & 

pa stan & 

pe & 
OPF(bu_cbl_ple_mth,argl) ~= plow 
} 


} 


Note 

1. No work environment check against actions ctrm, 
ctrb, ctrh, ctr, since these apply 
unambiguously to buried plant. 


2. Actions "bore" and “push” are not allowed here 
(i.e., to generate cable placement), since they imply 
only that structure work is to be done. If cable is 
placed in conjunction with this other. work, then the 
EPW mask should be used to enter multiple actions. 
endnote 


Fig. 9—Specification source file for STI 4P 13. 


4.2 Mechanical translation to design 


The objective of a mechanical translation to design is to take the 
specification from a file on the UNIX system Programmers’ Work- 
bench and after performing the appropriate transformations, submit 
the PL/1 source code corresponding to the specification for compila- 
tion via a Remote Job Entry (RJE) link. Once such a mechanical 
translation is realized, many problems at the object level can be 
resolved at the metalevel where they are often easier to deal with. 

The complete process is a shell procedure called spec to pli. 
Spec_to_pli performs the following steps in sequence: 

e Retrieve files—Retrieves the specification files. 

e Preprocess specifications—A buffer step to filter out any nota- 
tional variations in the specifications, and ensure that the input 
to the rest of the code generation process is ‘under control of the 
code generator. 

e Split out the component files—A single specification file may 
‘specify several decision tables. These are broken out in this step. 

e Alphabetize the Booleans—A housekeeping routine. To make the 
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object code more systematic, the logical variables corresponding 

to the condition codes in the decision tables are alphabetized. 

(There are often close to one hundred such variables.) 

e Shorten the lines—The code generator may use very long lines 
internally in its list of logical variables, but the PL/1 environment 
limits the line length to 72 characters. 

Rename files—An interface to match the file names to those 
expected by the decision table processor. 

Assemble files into decision tables—The four component parts of 
the decision table, described above, are assembled here. 

Generate PL/1 from decision tables—The decision table processor 
itself. 

Add JCL to PL/1 code—The job control statements needed by 
Time-Sharing Option (TSO) are added here. 

Cleanup—Remove miscellaneous working files that have been 
generated in previous steps. 

e Send to TSO—Submit to RJE. 

e Save the decision table and the PL/1 code—These items are 
stored in a separate directory, where they may be inspected, or 
printed out. 

Translation from a specification to a design requires the addition of 
those elements that are present in a design but are not present in a 
specification, or that are present in both but in a different form: 

e Variable declarations, defining the types of the variables, are 

generally not present in a specification; 

e Organization of sets of variables into a larger structure is present 
in a design because of programming or database considerations, 
but is not usually present in the specification. 

e If the design uses components that are referred to in the specifi- 
cation, but not described there, the design must solve the problem 
of resolving these references by including, or linking, the appro- 
priate code. 

e Occasionally, aliasing problems arise because of the preceding 
item, and the mechanical translation must deal with these. 

The solution to these problems is a program that constructs a 
structure to which the code in the decision table refers. In this 
structure, the organization, types, and specific names of the variables 
as seen by the decision table are all under our control. While it would 
be possible to write the code manually for the procedure to populate 
this mediating structure, it is rather simple to generate both the 
declaration of the structure to be populated and the program to fill it 
from a small data dictionary maintained for this purpose. This data 
dictionary can be a rather simple, ad hoc facility since it is designed 
for a rather limited purpose. 
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4.2.1 Current status of the code generator 


The current version of the code generator accepts as input the 
specification files, in the formatted version that is the customer copy 
of the specification. This is a propositional logic description of the 
conditions that generate the appropriate work credits—and, as noted 
above, is essentially an inverted decision table. The 20 specification 
files comprise 850 pages. The outputs of the code generators are 41 
decision tables and their associated PL/1 programs, since most of the 
specifications produce more than a single decision table. The PL/1 
code is approximately 16,500 lines of source code, the lines being 
relatively densely populated. Both the decision tables and the PL/1 
code are stored in files, with the PL/1 being sent for compilation as 
well. 

The code generator running time is about 0.1 second/line of system 
time, and 0.3 second/line of user time. In off hours the elapsed real 
time is about 0.6 second/line. Thus it is quite reasonable to regenerate 
the PL/1 code overnight if necessary. 


4.2.2 Productivity 


The literature on programming productivity describes several high- 
productivity methods for generating programs. Foremost among these 
are two techniques: reusable code and code generators. The reuse of 
code is applicable when one is developing a set of programs for similar 
applications, and common functions can be identified among the 
various applications. In that case the first application in the set to be 
developed pioneers the code, and subsequent applications reuse it. If 
the subroutines cannot be reused without modification, then they 
should be generalized in the hope of anticipating future requirements. 
Thus after several iterations, a considerable library of common func- 
tions and subroutines can be developed. The foremost example of 
reusable code is the extensive scientific subroutine package of Fortran. 

In the present application, the cases of enumerative complexity are 
sufficiently specialized that it does not seem likely that any other 
application would have need for these particular specifications. For 
this reason we have chosen to use the techniques of code generation 
here. 

As far as programmer productivity, a productivity rate in excess of 
ten thousand lines of tested code per year is readily achievable using 
code generation techniques. Indeed, the goal of code generation tech- 
niques is to ultimately put the programmer in the position of only 
developing and maintaining the code generators. In that case, appli- 
cations experts would retain all of the knowledge about the application 
and would generate formal specifications that could be compiled 
directly. Indeed, the purpose of formalization is to abstract the seman- 
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tics of the problem so that the programmer can deal with it as a purely 
syntactic problem. 

The code generation time required to produce all of the decision 
tables and associated PL/1 code for one 62-page specification was 1.5 
minutes of system time—4.5 minutes of user time—in a UNIX system 
on a VAX 11-780 computer.* The four decision tables generated in 
this case were 350 to 400 lines each. 

Using this code generator, it was possible to generate a completely 
fresh set of decision tables and PL/1 code each time that the specifi- 
cations were changed. In this way the specifications did indeed become 
the primary program description. 


V. ASSESSMENT 


The methodology and tools described in this paper were effective in 
producing a complex product on schedule, with the assurance that the 
program corresponded to the specification. Indeed, most of the speci- 
fication files were revised, some of them extensively, within the final 
week prior to delivery. When such revisions were necessitated because 
the specification did not accurately state the conditions for various 
work credits, the production code was regenerated from the revised 
specification. This has the advantage that the last-minute changes to 
the code are not patches, and avoids the problem of introducing new 
bugs when fixing old ones. 

The development of the module described in this paper supports the 
thesis that increasingly large roles in the software-development proc- 
ess should be played by code generation techniques. Indeed, the major 
problems encountered in the process of producing the code arose 
because we did not have some tools that should be part of the code 
generation facility. Examples are 

e Syntax checking—There was no syntax check of the specification. 
Syntax errors in the specification showed up only as compile-time 
or run-time errors. 

e Semantic checking—No tests were run to determine the correct- 
ness of the specification, until it was translated into production 
code. Most of the errors that were detected in the operational 
tests of the software were errors that could have been found by a 
diagnostic tool capable of executing the specification. 

The code generation process also provided severe tests of other 
software. The symbols and data structures that were generated ex- 
ceeded the capacity of some of the available tools. Examples are 

e Variable names are constructed automatically by the code gener- 


* Trademark of Digital Equipment Corporation. 
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ator and are semantically significant as an aid in diagnosis. Thus 
the statement in the specification: 


pacclass =’c’ 
generates the logical identifier: 
pacclass_eq.c. 


In some cases, this translation yields symbols that exceed the 31- 
character PL/1 limit on identifiers. (In those cases, provision was 
made in the generator to shorten the names. It was necessary in 
those cases to take special precaution to avoid name conflicts 
where two distinct names might have the same 31-letter prefix.) 
Lines constructed by the code generator often contained the entire 
specification of the conditions for a work task. In unabbreviated 
form these lines might be as long as 1,500 characters. Since awk 
is one of the primary tools used by the code generator and has a 
line length limit just under 512, this caused a problem. The 
solution was to use data compression and expansion at different 
stages in the generation process. (The primary tool used to effect 
the data compression and expansion is sed—the UNIX system 
stream editor.) 

In summary, the development of formal specifications and their 
automatic processing by computer yielded significant dividends in the 
JMOS project. Considering that the technology of application software 
generation is still quite new, we expect that much greater dividends 
can be obtained as we learn how to manage and organize software- 
development projects to more fully exploit this technology. In addition, 
new and improved tools and techniques will arise from aspects of the 
code generation process that are different from the manually produced 
code. 

Although we attempted to quantify the productivity gains that result 
from the processes that we have described, we feel that the technology 
is too new and there are too many variables to support particular 
claims. Much of the time of the authors was spent in developing tools 
and learning how to use them. Still a significant improvement was 
obtained in both the quality and quantity of the specifications and the 
delivered code, even when no allowances are made for the time spent 
developing the tools and learning how to use them. 
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The BX.25 link-layer protocol is a standardized procedure for establishing 
and maintaining a connection across a data link. Virtually all such standard- 
ized communication protocols worldwide are defined through English-language 
specifications. While the English-language BX.25 specification was developed 
with exceptional care and in fact represents an improvement over the inter- 
national standard X.25, it is argued here that it, or any English-language 
specification, can be expected to fall short of the implicit objective: to define 
implementations that will be mutually compatible and satisfy given perform- 
ance criteria. An alternative formal specification is presented. Among its 
attributes are that it is by its nature mathematically precise; it provides a 
medium for formal structured development and formal proof of performance 
of a task (validation) independent of any implementations; and the formal 
specification may be implemented into software or hardware in an automated 
fashion, reducing the need for laborious and repetitive implementations and 
virtually eliminating the need for “certification” of implementations. The 
specification given here can also be used to resolve issues of vagueness, 
ambiguity, and contradiction as they arise in the English-language BX.25 
specification standard. 


I. INTRODUCTION 


The BX.25 link-layer (level 2) protocol is for data packet inter- 
change between DTE (Data Terminal Equipment) customer equip- 
ment and DCE (Data Circuit-Terminating Equipment) network equip- 
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ment, or between two DTEs, over a single data link (a general reference 
for the terminology used here is Ref. 1). The protocol is designed with 
the expressed intention to be compatible with the International Stand- 
ards Organization (ISO) Standard High-level Data-Link Control 
(HDLC) and International Telegraph and Telephone Consultative 
Committee (CCITT) Recommendation X.25 Link Access Procedure 
B (LAPB). 


1.1 Function of level 2 


The functional purpose of level 2 is to provide end-to-end transmis- 
sion of “frames” (packets) with the capabilities of error and flow 
control. Error control is provided through error detection and error 
correction. Error detection is accomplished by means of a Cyclic 
Redundancy Check (CRC) for detection of erroneous frames, and by 
level 2 issued sequence numbers for detection of lost frames. Error 
correction is accomplished by retransmission of lost or erroneous 
frames. Flow control is provided through interstation signaling, which 
interrupts the generation of frames at the remote end. The protocol is 
defined for use on a synchronous, full-duplex link. 


1.2 OSN specification of level 2 


This paper is based upon the following document, hereinafter re- 
ferred to as OSN: “Link-Layer Logical Interface,” Section 2, Opera- 
tions System Network Protocol Specification: BX.25, Issue 2, Part II, 
published by Bell Laboratories in March 1980. (Since the writing of 
this paper, a third issue has appeared; however, the Part II, Section 2 
of concern here remains substantially the same in the new issue.) 


1.2.1 Purpose 


The purpose of OSN, as described elsewhere in the BX.25 document, 
is to provide a standard that will serve two goals. The first is to assure 
compatibility of two stations adhering to the standard. The second is 
to assure the performance of any implementation adhering to the 
standard. 


1.2.2 Means 


The means by which OSN sought to fulfill the above purpose is a 
very carefully formulated English-language description with numbered 
paragraphs and associated figures and tables (see OSN) that covers 
“every aspect” of operation of the protocol. The description is meant 
to describe the functions of the protocol, independent of any particular 
implementation. 
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1.2.3 Critique 


The BX.25 link-layer protocol is a standardized procedure for es- 
tablishing and maintaining a connection across a data link. The 
protocol is defined through an English-language specification, the way 
virtually all communication protocols are defined worldwide. In fact, 
the BX.25 link-layer protocol is a refinement of X.25 LAPB, a protocol 
adopted by the international telecommunications standards organi- 
zation, CCITT. BX.25 is compatible with X.25, and corrects certain 
problems associated with X.25. 

Why write a protocol specification? One answer might be to provide 
implementors with a conceptual sense of the facility to be imple- 
mented. This is a reasonable use for such a document, and many of 
the carefully worded English-language specifications previously 
adopted as standards, including BX.25, serve fairly well in this capac- 
ity. 

However, there is no guarantee that an implementation whose 
design is guided by such a specification will be compatible with any 
other such implementation. Nor is there a guarantee that any such 
implementation will maintain any level of performance whatsoever. 
When stated this way, there are few who would disagree. Nonetheless, 
there is an implicit inference held by many people that if a commu- 
nication protocol implementation “adheres” to a specification, then it 
will be compatible with all other “adhering” implementations and, 
furthermore, will maintain a certain level of performance. An overrid- 
ing difficulty with this inference, as we shall see, is that in practice 
there is virtually never a reasonable criterion for “adherence” of an 
implementation to an English-language specification. 

At the root of the difficulty is the semantic meaning of “specifica- 
tion.” Actually, specification would seem to entail a three-fold purpose, 
which goes far beyond the (certainly useful and viable) role of pre- 
senting a conceptual sense of a facility. According to this expanded 
purpose, a specification first of all should define (logically) a class of 
implementations (namely, those which adhere to it). Within the con- 
text of the specification, it should be possible to state a task that each 
implementation is intended to perform (e.g., maintain some level of 
performance). The specification should be such that it is possible, at 
least in theory, to determine whether it is logically consistent with the 
stated task (e.g., if part of the required performance is to detect all 
sequence number errors, it should be possible to determine by theo- 
retical means whether every adhering implementation necessarily will 
detect all sequence number errors, without actually testing each of 
what is most likely an infinite number of such possible implementa- 
tions). If it is not presumed that a task can be stated, or that the 
question of whether a task always will be performed is logically 
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meaningful, then adherence to a specification loses all practical sig- 
nificance. 

The question is whether an English-language specification can 
satisfy these three points. The claim here is that it cannot. The basic 
problem with English-language specifications is that they generally 
lack any concrete form or structure (such as one does find in “formal” 
specifications). This amorphous quality of English-language specifi- 
cations leads to a generality which engenders vagueness and ambiguity. 
Furthermore, a lack of structure fosters the appearance of inconsist- 
encies. 

For example, the BX.25 specification is the result of careful thinking 
by many talented individuals. It is well organized and formulated, 
probably as much so as any English language specification can be. It 
represents an improvement over the X.25 specification, which itself 
was the result of careful thinking by other individuals at the interna- 
tional level; BX.25 itself has undergone two major revisions. All this 
effort notwithstanding, BX.25 is nonetheless vague (e.g., the condi- 
tions under which a poll may be sent are never made explicit), 
ambiguous (e.g., precedence of the rejection of an out-of-range N(R) 
described in OSN 2.4.7, over the packet acceptance described in OSN 
2.4.5.2, while probably intended, is never stated), and contradictory 
(e.g., OSN 2.3.4.3 provides for retransmissions of REJ [Reject] while 
OSN Table 2.1c uses RR [Receiver Ready] consistently in its place). 
Given the quality effort that went into this specification, it might 
. seem unreasonable to place the blame for these deficiencies upon the 
individuals who created the specification; an alternative culprit is the 
specification medium: the nonformal, unstructured English language. 
This view is reinforced by the (now widely accepted) observation that 
such difficulties are not the exception but the rule in English-language 
specifications of communicative protocols. 

Definitions that are vague and ambiguous have limited use. It is not 
surprising to hear reports that different implementations of OSN (and 
virtually all other standardized communication protocols, for that 
matter) have been incompatible and of varying performance, even 
though this was the very situation OSN was intended to prevent. 

Let us suppose for the sake of argument that a protocol specification 
such as OSN were free of imprecision. Where would that leave it with 
respect to its purpose? How would one determine whether a particular 
implementation adheres to the specification? Since the specification 
is nonformal, a formally defined adherence criterion would have no 
meaning beyond its informal interpretation. Possibly the best adher- 
ence criterion would be that several experts examine the implemen- 
tation and declare that it adheres. The issue of stability of such a 
criterion aside, however, the consensus of experts is a patently im- 
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practical criterion, as many implementations involve optimized code 
of an arcane nature understood only by its designer. More likely, one 
would skirt the adherence criterion altogether, relying instead upon 
testing the implementation. What about testing (which is commonly 
called “certification” ?? It is a notoriously difficult problem to design a 
test for performance of a task. At best, such a test can only say with 
a certain level of confidence, assuming the implementation has certain 
properties (which, in fact, we can never know for sure that it does 
have), that it performs the task. In practice, however, it is virtually 
never the case that a testing procedure is sufficiently well analyzed 
that one can assign to the result a qualitatively defined degree of 
confidence. Thus, the test itself can only suggest that an implemen- 
tation is okay. Moreover, such a test certainly can say little or nothing 
about the overall integrity of the specification, since the relationship 
between the specification and an implementation is not quantized. (As 
a case in point, there have been satisfactory implementations of BX.25 
level 2, the above-mentioned deficiencies in the specification notwith- 
standing.) 

There are, in fact, models that are used for protocol validation, such 
as Refs. 3 and 4 (more about them later). However, to use such a 
model, a given specification such as OSN must be translated into the 
context of the model. As there is no unique way to do this, one might 
end up with a validated translation of OSN, but certainly not with a 
validated OSN (other translations may not be valid). 

Thus, the status of the English language as a specification medium 
is this: it does not lend itself to precise, unambiguous, or internally 
consistent specifications; and there is no meaningful way either to test 
(abstractly) the properties of a specification or to determine whether 
a particular implementation adheres to a specification. It does provide 
a conceptual sense of a facility. However, if this is the only requirement 
placed on a specification, the specification could be considerably 
simplified. For example, the concept of the link layer of X.25 is 
presented in Ref. 1 in a mere four pages (as compared to 27 pages for 
OSN). When one speaks privately to implementors who have imple- 
mented a protocol from an English-language specification, one often 
learns either they did not concern themselves with the precise details 
of the specification, or if they did, then these details have led them 
astray. 


1.2.4 This paper 


This paper presents an example of an alternative approach to 
protocol specification. The approach is to present a specification in a 
formal, mathematically based context that, by its very nature, renders 
the specification free of vagueness, ambiguity, and inconsistency. 
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Furthermore, this formal context not only provides a medium for 
precise specifications, but also provides a medium for a formal, struc- 
tured development of the protocol and for statement and formal proof 
of the performance of a task (independent of any implementations). 
The formal specification may be stored on-line as a database in a 
computer, and tested and altered quite easily, prior to any implemen- 
tations. Finally, the specification medium is such that a protocol 
specification may be implemented into software or hardware in an 
automated fashion. 

This paper presents a formal specification of BX.25 level 2, pre- 
sented through such a formal specification medium (called the selec- 
tion/resolution model for concurrent processes). The specification is 
intended to illustrate that such a formal specification can be quite 
readable (the implementors with whom I have interacted have been 
able to follow such a specification after about an hour of coaching). A 
production quality, formal specification of the protocol should be 
preceded by an informal introduction that presents the concept of the 
protocol (e.g., the above-mentioned four pages in Ref. 1). It is proposed 
that such a formal specification with a suitable introduction replace 
currently accepted English-language specifications. Not only would 
this permit true validation (formal proof of performance of task); it 
could also eliminate the need for laborious and repetitive implemen- 
tations of the same protocol, through application of automated imple- 
mentation. This would save time and effort, virtually eliminate the 
need for certification of implementations, and reduce the problem of 
interimplementation compatibility to the level of hardware interfaces. 

The specification given here can also be used to resolve issues of 
vagueness, ambiguity, and contradiction as they arise in OSN. Devia- 
tions from OSN occur only for the purpose of resolving inconsistencies 
as they appear in that specification; they are indicated by a comment 
in the right margin of the formal specification. In every case that the 
formal specification clarifies an ambiguity or resolves a vague issue in 
OSN, this clarification has been cleared with at least one expert on 
BX.25 level 2 (such clarifications were too numerous to indicate them 
in the formal specification). 


1.3 Other possible level 2 specifications 


There is no dearth of models for specification and analysis of 
concurrent processes [see the Special Issue on Computer Networks 
and Protocols of IEEE Trans. Commun., 28, No. 4 (1980)]. However, 
there is something unsatisfying about each such model I have exam- 
ined. Without trying in any way to give a critique of the literature, I 
will voice my complaints about two such models. The complaints (not 
the models) may be considered to be representative. 


564 TECHNICAL JOURNAL, FEBRUARY 1985 


Bochmann and Chung use a Petri net model for a synchronous 
environment in which transitions are determined by enabling predi- 
cates “specified in terms of a high-level programming language such 
as Pascal.”* However, this makes no sense in an asynchronous envi- 
ronment as the lower-level interactions of synchronization, interrup- 
tion, message exchange, and other “critical section” problems are 
hidden from view by the programming language. This aside, they 
describe no algorithm for combining several processes, a necessity for 
any analysis. 

Zafiropulo et al. use a finite state model based upon message 
exchange.* While this is (theoretically) capable of specifying processes 
in asynchronous environments and an algorithm for combining several 
processes is described, transitions are based simply upon the receipt 
or transmission of a message. There is no provision for complex 
interdependencies among processes such as exist in BX.25. 

Other specification formats such as Ref. 5 provide actual software 
to establish a database containing a well-defined, formal specification. 
However, no tools exist to analyze the resulting protocol. 


1.4 The selection/resolution model 


The format used here to specify level 2 is based upon a general 
model for concurrent processes.°” It is arithmetic in nature and has 
the property that the collective specification and joint behavior of 
several processes is computed by a fixed algorithm from the specifi- 
cations of the component processes alone. Processes are described in 
terms of states and enabling predicates. The states need not be listed 
exhaustively, but may be represented by one or more parameters. For 
analysis, a task is formally defined. To facilitate validation (proof that 
a given task is performed) reduction algorithms are available that 
substitute for a given system a simpler system with the (provably) 
same performance. Such reductions are necessary to combat intract- 
ability produced by “state explosion.” 


1.4.1 Methodological attributes 


The approach of the selection/resolution model has several meth- 
odologically desirable attributes. First, all algorithms may be applied 
mechanically without understanding the meaning of the protocol 
specification or task. Second, the procedure for defining the formal 
specification requires a complete systematic detailing of process be- 
havior, reducing the opportunity, which exists in a less organized 
approach, to overlook events. The significance of this is not negligible. 
Most approaches to specification suffer not so much through incorrect 
assertions as through what is overlooked. While specification mistakes 
and oversights are still possible in the model described here, they are 
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easily caught (see Section 1.4.2). The given specification is guaranteed 
to be logically complete in the sense that there can be no unexpected 
event. 


Not only does the specification provide a vehicle for analysis vali- 
dation, it provides the basis for implementation as a structured pro- 
gram. This is helpful not only to the implementor, whose programming 
job becomes straightforward if not trivial through automated imple- 
mentation, it is essential to assure the carry-over of compatibility and 
performance to each implementation. 


1.4.2 Quick review of the model 


A system is decomposed into many small interacting automaton- 
like processes. Suppose process A is in an OFF state and process B is 
in its READY state. The process C comprised of the two processes A 
and B taken together is then in a joint state (OFF, READY). Process 
A, let us suppose, has an ON state and a labelled directed edge from 
OFF to ON. The label, say, corresponds to the assertion, “A is in its 
OFF state and B is in its READY state.” This label encodes the 
condition under which the transition from OFF to ON may occur. It 
is denoted thus: 


(A:OFF)-(B:READY) . 


Suppose B has a labelled directed edge from READY to its state 
WILLING with the label 


(B:READY)-(A:ON) . 


A B 

OFF READY 

| (A:OFF) -(B:READY) )(B:READY)-(A:0N) 
ON WILLING 


Then the joint process C has an “edge” from its state (OFF, READY) 
to its state (ON, WILLING) with the label 


(A:OFF) -(B:READY)-(B:READY).-(A:ON) 
(The Boolean product of the label of A and the label of B). 


When interpreted as a Boolean expression, this has the logical value 
0 (always false) since (A:OFF) and (A:ON) can never be true simul- 
taneously. Thus, in fact, C has no edge between (OFF:READY) and 
(ON:WILLING). Had B’s label been rather 


(B:READY)-(D:?) 
then C’s edge would have had the (reduced) label 
(A:ON)-(B:READY).-(D:?) . 
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In this example, each process selects (broadcasts to the other proc- 
esses) the name of its current state. This is a special case of the more 
general presentation but is typical of the processes defined for level 
oe 

The joint behavior of all the processes that comprise level 2 taken 
together is modelled by taking all such possible products as illustrated 
above. This procedure is curtailed (so as to avoid the need to consider 
the labels on some 10” possible edges) through formal reductions of 
the system.°® 


1.5 How the structural organization of the protocol was established 


The division of an environment (in this case, level 2) into component 
processes (see Fig. 1) is part discipline and part art. The general 
approach involves several passes over a founding document such as 
OSN, a set of notes or evolving ideas. In the first pass, the so-called 
explicit processes are identified. These are objects, parameters, rou- 
tines, and the like that are explicitly named, such as timers, counters, 
messages, and channels. Some are more obviously translated into 
processes than others. For example, a counter is a conceptually simple 
process that advances from state N to state N + 1 each time the thing 
it is counting arrives. A message is produced by a sending process 
whose states correspond to the message it sends. (The Control Pri- 
mitive process CP sends the primitive REJ when it is in state REJ.) 
A channel process (such as level 1) is represented as a buffer process 
which stores the message M it is transmitting while in its state M. 
Delay is modelled by the length of time the channel process remains 
in the state M. In this pass, each such explicit process is named and 
its states are defined (to the extent possible). 

Once the explicit processes have been identified, another pass is 
made over the founding document and the list of explicit processes 
and associated states. In this pass, implicit processes are identified. 
These are mainly processes which keep track of what is going on (and 
hence are dubbed place-keeper processes). For example, if the founding 
document says, “Transmit SABM and wait for acknowledgement,” a 
place-keeper process must be defined that will (say) start in state 1 
and then move to state 2 when SABM (Set Asynchronous Balanced 
Mode) is sent. This enables the protocol to determine if an appropriate 
reception is in fact an acknowledgement and not simply out of the 
blue. 

Having identified all the component processes (explicit and implicit) 
and their respective states, a third pass is made, this time over the 
processes, to define the selections of each process. A selection is a 
signal that each process broadcasts from a given state. In level 2 
almost every process simply selects the name of the state it is in. In 
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Fig. 1—BX.25 link-layer (level 2) protocol structural organization local end. 
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other words, it broadcasts the name of its state. (An exception is the 
transmission process, TR.) Finally, a fourth pass is made in which the 
edge labels of each process are defined in terms of the selections of 
the various processes. These edge labels are Boolean functions in the 
selections of various processes that describe the conditions under 
which the associated transitions occur. 


1.6 Sources of errors 


Having completed the specification of all the component processes 
comprising a protocol, one begins a refining procedure to eliminate 
errors. This is best facilitated when the specification resides in an on- 
line database. Four types of errors are possible: 

1. Errors in the founding document (OSN) 

2. Errors of inference from the founding document 

3. Errors in transcription (typographical errors) 

4, Errors in interpretation of implementation considerations. 

The first three types of errors will be caught (insofar as they hinder 
the formally stated task of the protocol) during the validation phase, 
during which one applies algorithms to the specification database to 
prove that the given task is satisfied (or not satisfied). If it is deter- 
mined that the task is violated, the source of the difficulty is tracked 
down, corrected, and the validation procedure repeated. This may be 
continued until a satisfactory specification is found. (Alternatively, 
there are algorithms for synthesizing a satisfactory specification.® 
However, in the present case, the result would be quite different from 
OSN.) Aside from verification that the task is performed, it is often 
useful to examine possible histories of parts of the protocol to deter- 
mine whether there are any behavior abnormalities (whose exclusion 
was omitted from the scope of the task). 

The fourth type of error is caught upon implementation by the 
programmer or chip designer who notices that it is impractical or 
impossible to be faithful to the given specification. This is rectified by 
altering the specification accordingly and repeating the validation 
procedure. (An error of the fourth type would not arise if the specifi- 
cation were derived from an actual implementation rather than a 
conceptual design.) 


1.7 Synchronization 


The local end (level 2) and remote end are each assumed to be 
controlled by separate clocks driven by the respective clock of the 
associated incoming level 1 (see Fig. 2). Thus each level 2 is synchro- 
nized to its incoming level 1. The two ends are assumed to synchronize 
at the interface of level 2 and the outgoing level 1, but are otherwise 
asynchronous. (This is an inference, nowhere discussed in OSN.) 
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Fig. 2—BX.25 link-layer (level 2) protocol structural organization end-to-end. The 
two asynchronous components of an end-to-end link layer. 


In each synchronous end, time is divided into (not necessarily 
uniform) intervals. During each time interval (defined separately for 
each end by the centralized scheduler at each end), each process makes 
a selection (which in most cases is simply an announcement of its 
current state). Once each process has done this, and before the end of 
the time interval, each process resolves the selections of the other 
processes by moving to a new state along an edge whose label is 
enabled by the joint selections of the other processes. Once this is 
done, the time interval is ended. Since different edge labels have 
varying degrees of complexity, transitions will require varying lengths 
of time (and the associated time intervals will vary). All this is 
controlled by the scheduler for the corresponding synchronous end. 

However, unlike the specification format,° the state transitions all 
will be relatively quick, and thus it may be desirable to implement the 
time intervals so all have uniform length. This is especially true if the 
component processes are implemented in hardware. 


1.8 Purpose of this paper 


This paper serves to illustrate a specification of BX.25 level 2 using 
the selection/resolution model.®’ The specification was derived almost 
exclusively from OSN. While every attempt was made to avoid errors 
in inference, transcription, and interpretation, the greatest likelihood 
is that all three types will be found here. They, as well as errors in 
OSN, could be ironed out during validation, which will not be discussed 
further here. 


i. HIERARCHY OF PROCESS GROUPS 


The processes that comprise level 2 may be grouped according to 
function into a hierarchy of classes. At the highest level of the 
hierarchy, I identify three classes: 

1. Level 1 frame interface 

2. Internal memory 
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3. Level 3 packet interface. 

The level interface processes transmit or receive, and buffer signals or 
packets/frames between levels (i.e., packets between level 3 and level 
2, and frames between level 2 and level 1). Processes in the internal 
memory class include counters, timers, and place-keeper processes to 
remember what is happening, such as “link setup in progress.” The 
three main classes are further subdivided as follows, with process 
name acronyms indicated in boldface. 


1. Level 1 interface 


1.1 Outgoing 
1.1.1 Header Assembler 
1.1.1.1 C/R (Address)—identifies frame as command or re- 
sponse. 
1.1.1.2 CP (Control Primitive)—sets one of I; RR, RNR, REJ, 
SABM, DISC, DM, UA, FRMR. 
1.1.1.3 V(S)—sets the “send” sequence number N(S) (generated 
locally) of the next (information) frame to be transmit- 
ted. 
1.1.1.4 P/F—sets the poll/final bit. 
1.1.1.5 V(R)—sets N(R), the receive sequence number (gener- 
ated remotely) of the next expected information frame to 
be received. 
1.1.2 TR (Transmitter)—notifies outgoing level 1 and “internal mem- 
ory” processes of onset of frame transmission. 


1.2 Incoming (frame buffer) 
1.2.1 C/R’ (Incoming address buffer) 
1.2.2 CP’ (Incoming control primitive buffer) 
1.2.3 N(S) (Incoming N(S) sequence number buffer) 
1.2.4 P/F’ (Incoming poll/final bit buffer) 
1.2.6 N(R) (Incoming N(R) receive sequence number buffer) 
1.2.6 I’ (Incoming packet number buffer) 


2. Internal memory 


2.1 Counters 

2.1.1 UNACK—the next expected receive (acknowlegement) se- 
quence number N(R) (sequence generated on this end) in a 
received (information or supervisory) frame (0 < N(R) < 8). 

2.1.2 RETR—number of (re-) transmissions N of a frame subsequent 
to the expiration of timer T1 (0 < N < N2). 

2.1.3 PP (Packet Pointer)—points to packet in packet buffer that 
was last transmitted to level 1. 


2.2 Timers 
2.2.1 T1—times unacknowledged frame, pending initiation of retrans- 
mission procedure. 
2.2.2 T2—times period of link idleness pending declaration that re- 
mote station is unresponsive or inoperative. 


2.3 Place-keepers 
2.3.1 STS (Connection Status—States) S1 = disconnected 
S2 = link setup 
S3 = frame rejected 
S4 = disconnect request 
$5 = information transfer 


LINK LAYER PROTOCOL = 571 


S6 = REJ primitive sent 
S7 = waiting acknowledge- 
ment 
2.3.2 LS (Local host Status)—indicates status of local end. 
2.3.3 RS (Remote host Status)—indicates busy/not busy status of 
remote end. 
2.3.4 N(S)A (N(S) Analyzer)—indicates valid/invalid status of re- 
ceived N(S). 
N(R)A (N(R) Analyzer)—indicates valid/invalid status of re- 
ceived N(R). 
2.3.6 SD (Sender)—indicates to-be-sent/already-sent status of header 
assembler. 
2.3.7 SW(TR Shadow)—marks the instant that the transmitter TR 
turns on or off. 


2.3. 


on 


3. Level 3 interface 


3.1 Packet Buffer—holds packets from local level 3; packet is represented 
by (local) level 3-assigned packet number. 
3.1.1 I—indicates number of oldest (local level 3) packet in buffer; in 
sequence 1, 2,---. 
3.1.2 Q—indicates queue length in packet buffer. 
3.2 R(Packet Receiver)—receives packet number of packets received and 
passed by level 2. 


Ill, PERIPHERAL LEVEL 2 EQUIPMENT 


The following pieces of level 2 equipment have no effect on protocol 
performance (relative to the level 2 task) and hence are not modelled 
here. Conceptually (and likely physically, too), they sit between the 
level 2 processes defined above and level 1. Those marked (A) (respec- 
tively (B)) sit between the level 2 outgoing (incoming) level 1 interface 
and outgoing (incoming) level 1. 

(A) Flag generator—prefixes every frame with a flag consisting of the 
sequence 01111110, and generates continuous flags between frames. 
(A) Frame checking sequence generator—appends a 16-bit Frame 
Checking Sequence (FCS) field to the end of every frame that provides 
a CRC for error detection. 

(A) Bit-stuffer—provides for transparency of the flag by inserting a 
0-bit after any sequence of five contiguous 1-bits in the address, control 
and FCS fields. 

(A) Frame-abort sequence generator—transmits seven contiguous 1- 
bits (without 0’s) when a frame in the course of transmission is aborted. 
(See comment in specification of level-1 Transmitter TR given in 
Section VII.) 

(B) Idle link detector—detects an input of 15 contiguous 1-bits; when 
detected, the incoming link is declared idle and timer, T2, is set; when 
a 0-bit is detected, the idle condition is cleared. If T2 expires in this 
mode, level 2 STS returns to a disconnected state, S1. (This is 
modelled by a disconnect request DISC retransmitted N2 times.) 

(B) Invalid frame discarder—discards a frame not bounded by flags, 
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containing an address field other than 00000001 or 00000011, or having 
fewer than 32 bits between flags. 

(B) Bit-destuffer—reverses bit-stuffing procedure between flags. 

(B) Flat sensor—identifies beginning of frame by sensing opening i lag 
(exactly six contiguous 1’s). This facilitates the definition of the 
address, control, and information fields whose components provide 
the elements of the frame buffer, and it serves to mark the arrival of 
a new frame. 

(B) CRC decoder—discards a frame which contains errors detected by 
CRC. 

(B) Syntax checker—discards a valid error-free frame which contains 
any of the following syntactical faults: 

e Control field unknown 

e Receipt of an information field in a noninformation frame 

e Information field exceeds maximum established length 

e Receipt of an information field not containing an integral number 

of octets, if this is not allowed. 

When such a frame is detected and discarded, an FRMR primitive 
is sent in response. (Such faults have no way of being generated w. th 
a valid implementation of levels 2 and 3 [short of the presence of 
undetected errors introduced by level 1]. However, this process is 
simulated by the N(R) analyzer process N(R)A which causes an 
FRMR to be sent if the N(R) of an incoming frame is out of range 
[not between the last received N(R) and the last transmitted sequence 
number N(S)]. This fault also has no way of being generated by a 
valid implementation of levels 2 and 3, short of undetected level 1 
errors, and serves to represent, for the purpose of end-to-end perform- 
ance analysis, all the possible syntax faults which result in a FRMR 
response.) 

(B) FRMR information field generator—generates the special infor- 
mation field which is returned with an FRMR frame. (This field has 
no function with respect to the operation of the protocol. It is used 
exclusively for diagnostic purposes external to the normal operation 
of the protocol.) 

(B) Frame buffer—buffers the frame address/control fields as they 
arrive from level 1, via the level 2 peripheral equipment listed above. 
The syntactically valid address/control fields are separated (by the 
frame stripper) into the five components listed below, as the bits 
arrive. Information bits (if any) may also be buffered while level 2 
evaluates the (earlier arriving) address/control fields of the frame and 
determines whether the information field should be passed to level 3 
or discarded. (See the packet receiver process R described in Section 
VII.) The address/control fields cannot be evaluated until they have 
completely arrived and the frame stripper has set the values of the 
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following five memory elements of the frame buffer: 

1. Address 

2. Control primitive 

3. N(S) 

4, P/F 

5. N(R). 

Once the values of all five elements are set, the frame stripper 
signals level 2 to evaluate the frame. However, the final determination 
about the frame cannot be made until the FCS field has been received, 
the frame has passed the CRC (error check), and the closing flag is 
checked as valid. (The physical delay associated with filling the frame 
buffer and checking for frame validity and correctness [CRC] is 
modelled here as part of the delay in level 1. There is no conceptual 
difference between a physical implementation in which a frame buffer 
is sequentially filled and then presented in toto to level 2 for evalua- 
tion, and a level 1 that receives the frame, pauses, and then presents 
the five frame elements [simultaneously] to level 2 for evaluation.) 
(B) Frame discarding by peripheral equipment—the effect of frame 
loss, which results when any of the (unmodelled above-described) 
peripheral equipment discards a frame, is modelled here by loss of the 
frame in level 1. Hence, each frame seen by level 2 as modelled here 
is assumed to be error-free, valid, and syntactically correct, except 
possibly for an out-of-range N(R). 


IV. LEVELS 1 AND 3 


In order to analyze the end-to-end peer-level performance of level 
2, the channel between the local and remote ends (the physical buyer 
or level 1) must be modelled. Furthermore, as the end-to-end perform- 
ance is stated in terms of delivery of level 3 packets, level 3 must be 
modelled to the extent of packet delivery to the adjoining level 2. 


4.1 Level 3 


Level 3 is modelled implicitly as presenting two signals to level 2: 

e Transmitting packet 

e Idle. 
These signals are not explicitly represented, but are presented as a 
nondeterministic filling of the packet buffer. (The actual level 2/level 
3 interface is user defined, and may vary from implementation to 
implementation. However, its definition is not required in order to 
analyze the end-to-end performance of level 2.) 


4.2 Level 1 


The end-to-end performance is analyzed in terms of packet through- 
put in the context of a local level 2, remote level 2, outgoing level 1 
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(from local end to remote end) and incoming level 1 (from remote 
end to local end). Of course, it is enough to define one level 2 and one 
level 1. , 

Level 1 is modelled as a frame buffer with delay, in order to absorb 
the delay in level 1/level 2 interfacing devices such as the CRC checker 
(error detection) and frame format checker. (These are technically 
part of level 2, but are not specifically modelled here since they do not 
affect the end-to-end behavior, aside from producing delay). The 
capacity of level 1 could be modelled as infinity. However, it can be 
shown on a theoretical basis that it will never hold more than k frames, 
where k is the maximum number of transmitted I-frames permitted to 
be unacknowledged by remote level 2. Furthermore, all the delays 
attributed to level 2 are smaller than the delay of transmitting one 
frame. (One frame is at least five octets or 40 bits long. For the purpose 
of this model, the delay of certain level 2 overhead, including error 
detection and frame format checking, has been attributed to level 1. 
This contributes a noncumulative delay of no more than the time 
required to transmit three octets.) Hence, level 1 may be modelled as 
a buffer with a capacity of only one frame, which is pushed out of the 
buffer by a nondeterministic selection change after an arbitrary delay. 
(The model thus models a situation which, in this respect, is more 
general than reality. There is no harm in doing this if the desired 
conclusions are nonetheless obtained. Presumably, we do not want a 
level 2 which is critically sensitive to timing considerations.) 

The processes of level 1 are designed as follows: 


Outgoing level 1 


. C/R1 address buffer 
CP1 control primitive buffer 
. N(S)1 N(S) buffer 
P/F 1 P/F buffer 
. N(R)1 N(R) buffer 
. I1 I buffer 
. R1 Receive moderator (indicates whether level 1 is ready to 
receive) 
8. S1 Send moderator (indicates whether level 1 is ready to send). 


NTH TRON 


Incoming level 1 component processes are denoted by acronyms 
followed by a prime (’). 


4.3 Synchronization 


Incoming level 1 provides clocking to level 2 through its bit stream 
and flag generation. This establishes bit and octet synchronization 
with level 2. Hence, the process comprising local levels 2 and 3, the 
local host and incoming level 1 are all assumed to be synchronized 
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(driven by a common clock), and are modelled as synchronous proc- 
esses. The local end is defined to be (the product of) these processes. 
Remote levels 2 and 3, the remote host and outgoing level 1 comprise 
the remote end. Thus end-to-end behavior is modelled by these two 
processes (the local end and the remote end) taken together. The 
stabilization’ of each of these two processes represents the two ends 
as asynchronous (general) processes. Intuitively, the “asynchronous 
connection” between the local end and the remote end is provided by 
the transmitter process TR of level 2, which synchronizes its trans- 
mission to level 1 by sensing the level 1 clock. This is modelled by 
permitting transmission to level 1 only in the presence of a level 1 
selection, “level 1-is-ready” (process R1), seen by the transmitter of 
level 2. 


V. LOCAL HOST 


The computer comprising the residence of local levels 2 and 3 (local 
host) may disconnect the node (for preventive maintenance, for ex- 
ample). This is done in an orderly fashion. Before the actual discon- 
nection, local level 2 informs the remote end that a disconnection is 
in progress. The actual disconnection takes place only once this notice 
has been acknowledged by the remote end. 

Similarly, to reestablish a link, the host causes the level 2 link setup 
procedure to be initiated: 

Furthermore, if local difficulties cause a temporary inability to 
process incoming frames (for example, a temporary malfunction which 
causes a buffer overflow condition), the local host may suspend incom- 
ing packets by issuing a busy signal. The effect of this is that local 
level 2 discards any incoming information fields and notifies the 
remote end of this condition (using the RNR primitive). When the 
condition terminates, the host clears the busy condition, and level 2 
in turn notifies the remote end (with an RR primitive). 

The interaction of the local host with local level 2 is modelled by 
providing the local host (process H) with four L-selections visible to 
local level 2, defined below. 


Host L-selection Interpretation 
(H:GO) Local host start command 
(H:STP) Local host stop command 
(H:B) Station becomes busy 
(H:NB) Busy condition clears 


Vi. SYSTEM PARAMETERS 


There are four system parameters upon whose definition this spec- 
ification depends. (There are other system parameters, such as maxi- 
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mum allowed size of information field or timer periods, upon which 
this specification does not depend.) The four parameters are: 
e Sequence number modulus (set at eight here and in OSN descrip- 
tion) 
e N2—maximum number of (re-) transmissions of a frame subse- 
quent to the exploration of timer T1 
e k—maximum number of transmitted I-frames permitted to be 
unacknowledged by remote level 2 (0 < k < 8) 
© Qmax—capacity (in number of packets) of level 3 packet buffer 
(0 < Qmax < 8). 


VII. PROCESS SPECIFICATION 


The link layer interface (level 2) is specified as a product of 27 
component processes. Additionally, the physical layer (level 2) is 
specified as a product of 16 component processes (8 incoming, 8 
outgoing). The processes, outlined in the previous section, are formally 
specified here. 


7.1 Format 


The format of specification is as follows. The comments column 
includes the source in OSN for the given edge labels. 


<process acronym> (<process number/name>) 


<verbal description> 


states: <list of state acronyms followed by (names)> 
<comments> 
selections: <list of L-selection acronyms followed by (names)> 


<state acronym (name) A>—> 
<state acronym a>: <edge label of the edge (A,a)> 


<state acronym (name) B>—> 
<state acronym a>: <edge label of the edge (B,a)> 


7.2 Special notation 


1. Unlisted edge labels are 0 (no edge). 

2. When the list of L-selection acronym (names) is identical to the 
list of state acronym (names), the former is omitted. 

3. When states are identified implicitly in terms of parameters, the 
range of the parameters is described in the comments column. 

4. The word “otherwise” is used to denote the complement of the 
disjunction (inclusive OR) of the other labels on edges outgoing from 
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the state in question, multiplied by the sum of the L-selections at that 
state. 

5. LN] denotes the smallest nonnegative integer congruent to N 
modulo 8 (in particular, [8] = 0); [t < m < n] denotes [1] < [m] < [n] 
if [¢] < [n], and denotes 0 < [m] S [n] or [i] < [m] <7 otherwise. 

6. If A is a process, its L-selections are expressed as (A:S) where S 
is the selection identifier. The standard relations’ are satisfied, namely, 
(A:S)(A:S’) = 0 if S 4S’, (A:S) and (B:S’) are independent if A# B 
and Ys (A:S) = 1. 

7. A prefix dot (@) in edge labels of edges outgoing from a state S of 
a process P denotes the L-selection (P:S). 

8. We may denote >)? (A:S;) = (A:{S,, --- ,S,}) when convenient. 

9. Each process has a single initial state, which is either the state 
listed first, or in the case of a parameterized state variable, the smallest 
value of that variable. 


7.3 Common elements 


The following are definitions of expressions which occur in the edge 
labels of more than one process. They may be used in implementation 
to centralize the redundant computations they present. Each expres- 
sion is defined in terms of a character a(<acronym>), headed in 
parentheses by the list of the process in which it appears, and meaning, 
if relevant. 


(C/R, PF Poll may be set) 

a(poll) = (N(R)A:OK)[(STS:(5,6,7))[(CP’:I)(LS:NB)(RS:NB) 

+ (CP’:{RR:REJ})]-(C/R’:C)(P/F’:0) + (C/R’:R)] + (H:GO) 

+ (LS:STP) + (RETR:N2) + (T2:EXP)(STS:3) 
(CP, P/F A final transmission is pending) 
a(fp) = (C/R:R)(P/F:1)(SD:C) 
(CP, TR, SD All packets in buffer have been sent) 
a(aps) = )) (PP:n)(Q:n) 


n 


(CP, RETR, T1, I No new acknowledgment of an outstanding I 
frame) 


a(nn) = » (UNACK:n)(N(R):n) 


(TR, T1 All sent I frames have been acknowledged) 
a(aa) = } (UNACK:n)(V(S):n) 
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7.4 Level 2 processes 


The 27 processes which comprise local level 2 are specified here. 
The processes which comprise remote level 2 are identical but that to 
each of their acronyms is adjoined the suffix “r”. (This may be seen 
in the edge labels of incoming level 1.) 

For the interconnection of these 27 processes, see Figs. 1 and 3. 


C/R (1.1.1.1 address assembler) 

Identifies a frame as a command or response by defining the address 
field. (For command [respectively, response], address is set to that of 
the remote [local] end.) The edge labels are taken from OSN Table 
2.1 and from the need to ensure that a pending “final” (i.e., a response 
with P/F bit 1) is not superceded as a result of a subsequent frame 
arrival, as described in OSN sections relating to the poll bit (see 
description of P/F process). 
states: C (command) 

R_ (response) 


(command) —> 
:-a(poll)[(T,:EXP) + (STS:3)] 
: otherwise 
(response) —> 
:-((P/F:0) + (SD:L)][a(poll) + (T):EXP)(STS:3)] 


: otherwise 


02 OC) 


CP (1.1.1.2 Control Primitive Assembler) 

Identifies command/response control primitive in the frame control 
field of the next frame to be sent. This may be updated several times 
before it is actually sent, if several frames arrive from level 1 during 
the course of transmission of a long information frame. Level 3 packets 
and send sequence numbers appear exclusively in frames containing 
the information control primitive I. Acronyms are those of OSN. Edge 
labels are from OSN Table 2.1 and 2.4.5.2b. 
states: I (Information command) Frames containing this con- 

trol primitive are called infor- 
mation frames and contain a 
send sequence number N(S) 
and a receive sequence num- 
ber N(R) in the control field 
and an information field con- 
taining a level 3. 

RR (Receiver Ready) Indicates local station is 
ready to receive an I frame; 
frame contains N(R) in the 
control field (acknowledging 
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RNR (Receiver Not Ready) 


REJ (Reject) 


SABM (Set Asynchronous Bal- 


previously sent I frames). “Su- 
pervisory” frame. 

Indicates host is busy (un- 
able to accept packets); frame 
contains N(R). 

“Supervisory” frame. 

Frame contains N(R) (ac- 
knowledging previously sent I 
frames); constitutes request to 
retransmit frames with send 
sequence numbers = N(R). 

“Supervisory” frame. 

Indicates host wants to set 


anced Mode Command) up link. 


DISC (Disconnect Command) 


DM _ (Disconnected Mode 
Response) 

VA (Unnumbered Acknowl- 
edgment Response) 

FRMR (Frame Reject Re- 


sponse) 


Indicates host is discon- 
necting from link (e.g., for 
prevention maintenence). 

Indicates host is discon- 
nected from link. 

Indicates receipt of SABM 
or DISC. 

Indicates receipt of semant- 
ically bad frame (e.g., N(R) 


out of range). 


Set a = (CP’:I)(P/F’:0)(STS:{5,7})(LS:NB)(RS:NB) 
+ (CP’:{RR,REJ})[(C/R’:C)(P/F’:0) 
+ (C/R’:R)](STS:{5,6,7}), 

6 = (LS:{NB,B})(RETR:N2)(N(S)A:OK)(N(R)A:OK) 

y = [(T1:EXP) + (C/R’:R)(P/F’:1)](T2:EXP). cf: OSN 2.4.5.8 
v(parameter) —> v € {I.RR, ..., FRMR} 
I: ss - (fp) a(aps) ay 
RR: -a(fp)B(STS:{5,8,7})[y{(CP’:{I,.RR,RNR,REJ})(P/F’:1) 

(LS:NB) + (CP’:I)(P/F’:0)(LS:NB)(RS:B) + a(aps)a} 
+ [(T1:EXP) + (T2:EXP)][(STS:5)(RS:NB) + 
(STS:6)(RS:B)](LS:NB) + (T1:EXP)(STS:7)(LS:NB)] 
: | The a(aps)a term is from OSN 2.4.5.1.2. 
-(RETR:N2)[(STS:{5,6,7})[(H:B)(LS:NB) + 
(LS:B)[(CP’:1) + (P/F’:1)(CP’:{RR,RNR,REJ}) + 
(T1:EXP) + (T2:EXP)]]] 
-a(fp)(LS:NB)(RETR:N2){[(T1:EXP) + 
(T2:EXP)](STS:{5,6})(RS:NB) +: 
(N(S)A:REJ)(STS:5)} 
-{(CP’:DM)(STS:1) + (CP’:{DM,FRMR})(STS:3) + 
(CP’:{DM,FRMR,UA})(STS:{5,6,7}) + 


RNR: 


REJ: 


SABM: 
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(T1:EXP)(STS:1) + [(T1:EXP) + (T2:EXP)](STS:2) 
+ (RETR:N2)(STS:{3,5,6,7} + (H:GO)} 

DISC:  -&(fp){(LS:STP)(STS:{1,4}) + [(T1:EXP) + 
(T2:EXP)](STS:4)} 

DM: __ -{(STS:1)[(CP’:{LRR,REJ,RNR})(C/R’:C)(P/F’:1) + 
(CP’:DISC)] + (STS:2)(CP’:DISC) + 
(STS:4)(CP’:SABM)} 


UDA: -[((CP’:SABM)(STS:4) + (CP’:DISC)(STS:{3,5,6,7})] 
FRMR: -(N(R)A:FRMR) 
v: otherwise 


V(S) (1.1.1.3 sequence number generator) 

Indicates the sequence number N(S) (assigned here) in the frame 
control field of the next information frame to be transmitted. This is 
“seen” by level 1 (i.e., is part of the frame transmitted to level 1) in I 
frames only. 


states: N (parameter) 0O<N<8 
N- 

[N+1] -(SW:0)(TR:ON)(CP:]D OSN 2.3.1.4.1 
M: -(CP’:REJ)(N(R):M)(N(R)A:OK) OSN 2.3.5.5,0<M<8 
0: -[(CO’:SABM) + (CP:SABM)}] if N # 0; OSN 2.3.3.5 
N: otherwise 


OSN refers to initialization of N(S) only 
upon receipt of SABM; presumably, this 
must be done by sending end as well, as 
modelled here. 


P/F (1.1.1.4 poll/final bit assembler) 

Indicates value (0 or 1) of poll/final bit in the frame control field of 
the next frame to be transmitted. (OSN is very vague about this. 
According to Fred Berg [private communication], the purpose of a poll 
[P/F bit set to 1 in a command frame] is to force the other end to 
respond to the associated command, rather than to wait and respond 
possibly to a subsequent command [or I frame], as is allowed if there 
_is no poll. The response is required to be a “final” [P/F bit set to 1 in 
a response frame], which indicates it is a response to the outstanding 
poll, of which there is allowed to be at most one [from each end] at 
any time. However, when to send a poll is, with one exception, nowhere 
specified in OSN and thus is left to the implementer. The exception 
is that when timer T1 expires, a poll is required [OSN 2.4.5.8]. Hence, 
it must be the intention of OSN that the protocol be equally valid, 
independent of the extent to which polls are ever used. Furthermore, 
this polling convention leads to certain inefficiencies. For example, 
the response during the information transfer to a supervisory poll 
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must be an RR [final], whereas an I frame would convey the same 
information [less the final bit] more efficiently.) The following sections 
of OSN mention the P/F bit: 2.3.2, 2.3.3.2, 2.3.3.3, 2.3.3.4, 2.3.3.8, 
2.3.4.2, 2.4.3, 2.4.4.4, 2.4.4.6, 2.4.5.5, 2.4.5.7, 2.4.5.8, 2.4.7, 2.4.8.1. 


states: 0 Outgoing frame not poll/final. 
1 Outgoing frame is poll or final. 
v (parameter) —> v € {0,1} 


1: -[(C/R’:C)(P/F’:1)(N(R)A:OK) + a(fp) 
+ (STS:3)(T1:EXP) + a(poll)] 

0: -[a(fp)[(C/R’:R) + (P/F’:0)] + a(poll)] —a(poll) is added to both 
edges to model polling 
conventions for all legal 
implementations. 


TR (1.1.2 transmitter) 
Indicates to level 1 the on/off status of the frame transmission. 
When the transmitter is on, level I receives sequentially the bits 
comprising a frame, including those of the address/control fields and 
the information field if the frame is an I frame or FRMR frame. When 
the transmitter finishes to send a frame, it turns off and one or more 
flags are then (automatically, by unmodelled peripheral equipment) 
sent across level 1. The periods during which the transmitter is on can 
be of variable length, depending upon the length of the information 
field and the amount of bit-stuffing required. This is modelled by a 
nondeterministic state change from the “on” states to the OFF state. 
The lengths of the periods when the transmitter is off must be an 
integer multiple of the length of time required to send one flag, i.e., 
one octet (8 bits). This is modelled by a level 1 state change from the 
“not ready” state to the “ready” state which is assumed to occur at 
each instant that a complete flat has been sent. 
The transmitter process also optionally aborts the transmission of 
an I frame when a REJ primitive is received concurrently (in an error- 
free, valid, syntactically correct frame). The delay associated with the 
transmission of the frame abortion signal by the peripheral frame- 
abort-sequence generator is modelled here as if absorbed in the delay 
until the “on” state is next reached. 
states: OFF 
ON/I (transmitting I frame) 
ON (transmitting non-I frame) 
AB (abort) 

selections: (TR:OFF) 
(TR:ON) 
(TR:ON) 
(TR:AB) 
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OFF — 

ON/I: -(SD:C)(CP:I)(R1:1) 
ON:  -(SD:C)(CP:1)(R1:R) 
OFF: -otherwise 


V(R) (1.1.1.5 received sequence number assembler) 
Indicates the sequence number N(S) (assigned at remote end) in 
frame control field of the next expected information frame to be 
received. This is “seen” by level 1 (i.e., is a part of the frame trans- 
mitted to level 1) in I frames and supervisory frames (RR, RNR, REJ) 


only. 

states: N (parameter) 0<N<8 
N- 

[N+1] - (CO:D(N(S):N)(N(R)A:OK) OSN 2.3.1.4.3 
0: - [(CP:SABM) + (CP:SABM)] OSN 2.3.3.5 
N: otherwise 


OSN refers to initialization of V(R) only 
upon receipt of SABM; presumably, this 
must be done by sending end as well, as 
modelled here. 


ON/I (transmitting I frame) > 


OFF: (TR:ON) OSN 2.4.5.5b 

AB: (TR:ON)(N(R)A:OK) Abort is optional. 
(CP’:{REJ,SABM}) 

ON/I: (TR:ON) OSN does not describe 


abort upon receipt of 
SABM: added here. 


ON: (transmitting non-I frame) > 


OFF: 

ON: . 

AB: (abort) > 

OFF: -a(aa) 
AB: otherwise 


C/R’ (1.2.1 incoming address buffer) 
Buffers the address field of a frame arriving from level 1. 
states: C (command) 
R (response) 

C (command) —> 

R:-(C/R1’:R)(S1’:1) 

C: otherwise 

R (response) —> 

C:-(C/R1’:C)(S1’:1) 

R: otherwise 
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CP’ (1.2.2 incoming control primitive buffer) 
Buffers the control primitive in the control field of a frame arriving 
frame level 1. 
states: I For names of states, see control primitive assembler 
RR process 1.1.1.2. 


FRMR 
v (parameter) > u,w € {I,....FRMR} 
w:-(CP1’:w)(S1’:1) Ww ¥ Dv 
v: otherwise © 
N(S) (1.2.3 incoming sequence number buffer) 
Buffers the sequence number in the control field of a frame arriving 
from level 1. 


states: N (parameter) 0<N<8 
N (parameter) — 0<NM<8 
M (parameter):-(N(S)1’:M)(S1’:1) MAN 


N: otherwise 
P/F’ (1.2.4 incoming poll/final bit buffer) 
Buffers the poll/final bit in the control field of a frame arriving 
from level 1. 


states: 0 

1 
v (parameter) —> v,w & {0,1} 
w (parameter):-(P/F1’:w)(S1’:1) WFD 


v: otherwise 
N(R) (1.2.5 incoming receive sequence number buffer) . 
Buffers “receive sequence number” in the control field of a frame 
arriving from level 1. 


states: N (parameter) 0o<N<8 
N (parameter) > 0<N,M <8 
M (parameter):-(N(R)1’:M)(S1’:1) MAN 


N: otherwise 
I’ (1.2.6 incoming information field buffer) 
Buffers the packet number of the packet in the information field of 
a frame arriving from level 1. 


states: N (parameter) 0<N 
N (parameter) > 0<N,M 
M: (11’:M)(S1’:1) 0<N,M 


N: otherwise 
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UNACK (2.1.1 first unacknowledged frame number counter) 
Indicates the next expected receive (i.e., acknowledgment) sequence 
number N(R) in a received (information or “supervisory”) frame. 
Receipt of the number N(R) = N constitutes acknowledgment from 
the remote end that the (locally generated) information frame with 
sequence number N(S) = N was received by the remote end. 


states: N (parameter) O<N<8 
N (parameter) —> 0<NM<8 
M (parameter):-(N(R):M)(N(R)A:OK) MAN 


N: otherwise 


RETR (2.1.2 retransmission counter) 
Counts the number of (re-) transmissions of a frame subsequent to 
the expiration of timer T1. OSN 2.4.5.8 and 2.4.8.2. 


states: N (parameter) 0<NZ<N2 
N (parameter) > 0<N<N2 
N+1: - (T1:EXP) 

0: - [(CP:{UA,RNR}) + 


(CP’:{.RR,RNR,REJ}). 
(N(S)A:0K)(N(R)A:OK)a(nn)] 
N: otherwise 
OSN is vague about what reception is cor- 
rect enough to warrent resetting RETR to 
0 (e.g., RNR is accepted, according to OSN 
2.4.5.8, even with (N(R)A:FRMR) whereas 
acceptance of I with (N(S)A:REJ) is not 
indicated positively or negatively). 
N2—> 
0:-(CP:SABM)(SD:L) 
N2: otherwise 


PP (2.1.3 packet pointer) 
Points to packet in packet buffer that was last transmitted to level 
1. (This process “goes back N” for retransmission of lost packets. 
When acknowledgments arrive, it must decrement accordingly, to keep 
pace with the packet buffer queue length, process Q.) 


states: N (parameter) 0< NS Qmax 
N- 0O<N S Qyax 
N +1: -(SW:0)(TR:ON)(CP:1D if N < Qmax 
M: -(N(R)A:OK) > {(N(R):i)(UNACK:)) 
i-i]=N-M 
+ (CP’:REJ)(N(R):7)(V(S):7)} 
N: otherwise 


T1 (2.2.1 timer 1) 
Times unacknowledged frame, pending initiation of retransmission 
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procedure. (Expiration of timer is modelled as a nondeterministic 

selection change, and thus is more general than reality.) 

states: OFF 

ON 
EXP (expired) 

Define 

a = (CP:{REJ,SABM,DISC,FRMR}) + (C/R:C)(CP:RR) 

+ a(aa) + (C/R:C)(P/F:1) 

OFF > 

ON: - a@ 

OFF: - [a+(CO:FRMR)] OSN 2.4.4.1, 2.4.4.3, 2.3.3.3, 2.3.4.3, 
2.4.5.4, 2.4.8.1 and 2.4.6, in which set- 
ting T1 when sending FRMR is unfor- 
mal. In part, a is defined in accordance 
with the comment to state 6 of STS. 

Define 

6B = (STS:{2,4})(CP’:UA) + (N(R)A:OK)(CP’:{RR,RNR,REJ}) 

(C/R’:R)(P/F’:1) + a(nn) 


ON —> 

OFF: - £8 OSN 2.4.4.1, 
2.4.5.4, 2.4.5.8 

EXP: B 

ON: .- 8B 

EXP > 


OFF: - (TR:ON)(SD:L)[(STS:{1,2})(CP:SABM) + 
(STS:3)(CP:FRMR) + (STS:4)(CP:DISC) + 
(STS:{5,7})(LS:NB)(CP:RR) + (STS:6)(LS < 
NB)(CP:REJ) + (STS:{5,6,7})(LS:B)(CP:RNR) + 
(N(R)A:OK)(P/F’:1)(C/R’:R)a(nn)] 

EXP: otherwise 


T2 (2.2.2 timer 2) 

Times period of “link idleness” and periods during which consecutive 
flags are received. (Expiration of timer is modelled as a nondetermin- 
istic selection change, and this is more general than reality.) 
states: OFF 

ON 
EXP (expired) 
OFF —> 
ON: -(T1:0FF) 
OFF: otherwise 
ON > It is assumed that T2 times any combination 
of link idleness and consecutive flags (OSN is vague 
about this). 
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If the local end rejects a frame while the remote 
end enters a failure mode wherein it transmits 
continuous flags (only), the local end will eventually 
wait forever in STS state 7, without any further 
transmissions. 

EXP: -(T1:0N)(TR:OFF) 

OFF: -(T1:0N) + (TR:OFF)] 

ON: -(T1:0N)(TR:OFF) 

EXP > 

OFF: -(TR:OFF)(SW:0)(SD:L) 

EXP: otherwise 


STS (2.3.1 connection status) 

Indicates status of level 2 link control. Acronyms, names, and 
definition are from OSN Table 2.1. (Collisions of conflicting com- 
mands from remote end, host or internal devices such as timers are 
rarely resolved in OSN; here, in each case a [hopefully] reasonable 
resolution is given.) 
states: 1 (disconnected) 

2 (link setup) 
3 (frame rejected) 
4 (disconnect request) 
5 (information transfer) 
6 (REJ control primitive sent) 
7 (waiting acknowledgment) 
1 (disconnected) — 
2:-[(CP’:DM) + (T1:EXP) + (H:GO)](CP’:SABM) 
5:-(CP’:SABM) 
1: otherwise 
2 (link setup) > 
1:-(CP’:{DM,DISC})(LS:STP) 
4:.-(LS:STP)(CP’:UA) 
5:-(CP’:UA) 
2: otherwise 
3 (frame rejected) > 
1:-(CP’:DISC) 
2:-[(CP’:{DM,FRMR}) + (RETR:N2) + 

(H:GO)](LS:STP)(CP’:{SABM,DISC}) - 

4:.(LS:STP)(CP’:{SABM,DISC}) 

5: (CP’:SABM) 

3: otherwise 

4 (disconnect request) — 
1:-[(CP’:{UA:DM}) + (RETR:N2)](H:GO) 
2:-(H:GO) 

4: otherwise 
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Set a = [(CP’:{UA,DM,FRMR}) + (RETR:N2) + (H:GO)] 
-(CP’:DISC)(N(R)A:FRMR)[(LS:STP) + a(fp)]. 
6 = (N(R)A:FRMR)(CP’:DISC), 
= a(fp)(LS:STP)(CP’:DISC)(N(R)A:FRMR), 
= a(fp)[(T1:EXP) + (T2:EXP)](CP’:{DISC,UA,DM,FRMR}) 
(RETR:NZ)(H:GO)(N(R)A:FRMR)(LS:STP), 
e = a(fp)(LS:NB)(N(S)A:REJ)(N(R)A:OK) 
(information transfer) > 
:-(CP’:DISC) 
7° QE 


7-6 


5 

1 

2 

3 

4:-¥ 
6:-e(CP’:DISC) 

7:-(LS:NB)6e 

5: otherwise 

6: (REJ control primitive sent) > 
1:-(CP’:DISC) 
2:-a(CP’:{I,SSABM}) 

3:°8 

4 
5 


iy 
:-(LS:NB)[a(fp)(CP’:I) + (CP’:SABM)]. 
(N(R)A:FRMR)(CP’:DISC) 

7:-6(CP’:{I,SABM}) 

6: otherwise 
Modelled according to OSN Table 2.1.c 
which permits at most 1 retransmission of 
REJ, this contradicts OSN 2.3.4.3 which 
permits up to N2 retransmission of REJ. 
Here, RR rather than REJ is retransmitted 
upon expiration of T1, up to N2 times. 

7 (waiting acknowledgment) > 

:-(CP’:DISC) 

:-a(CP’:SABM)[(CP’:{RR,RNR,REJ}) + (C/R’:C) + (P/F’:0)] 

“6B 


Y 

:-[(CP’:SABM) + [a(fp)(CP’:{RR,REJ}) + (CP’:RNR)] 
(C/R’:R)(P/F’:1)] 

7: otherwise 


LS (2.3.2 local host status) 
Indicates status of local end as deduced from last received control 
signal from the local host. OSN Table 2.1. 
states: NB (not busy) 
B (busy) 
STP (stop) 
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OFF 
NB (not busy) > 
B: -(H:B) 
STP:-(H:STP) 
NB: otherwise 
B (busy) > 
NB: -(H:NB) 
STP: -(H:STP) 
B: otherwise 
STP (stop) > 
OFF: -(STS:1) 
NB. -(H:GO) 
STP: otherwise 
OFF — 
NB: -(H:GO) 
OFF: otherwise 


RS (2.3.2 remote host status) 

Indicates busy/not busy status of remote end as deduced from a 
receipt of RNR control primitive from remote level 2, based upon 
OSN Table 2.1. 
states: NB (not busy) 

B_ (busy) 
NB (not busy) > 
B: -(CP’:RNR)(N(R)A:OK)(STS:{5,6,7}) 
NB: otherwise 
B (busy) > 
NB: (CP’:{RR,REJ,SABM})(N(R)A:OK) 
B: otherwise 


N(S)A (2.3.4 N(S) analyzer) 

Indicates whether or not the sequence number N(S) of the last 
received (information) frame is the next expected sequence number. 
If not, it causes a retransmission request (REJ primitive) to be sent. 

OSN 2.3.4.2 

states: OK 

TEST 

REJ 
OK > 
TEST:.-(S1’:1) 
OK: _ otherwise 
TEST — 
OK:.- [(STS:{5,6,7}) + (CP’:1) + ¥ (V(R):n)(N(S):n)] 


REJ: otherwise 
REJ — 
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OK: -a(fp) 
REJ: - a(fp) 


N(R)A (2.3.5 N(R) analyzer) 

Indicates whether or not receive sequence number N(R) in the last 
received (information or “supervisory”) frame is “within range,” i.e., 
whether or not [UNACK* < N(R)* < V(S)*], where (*) denotes the 
state of the associated counter. An N(R) is within range if and only if 
it acknowledges a sent but yet unacknowledged frame (in which case 
it also acknowledges all previous unacknowledged frames). If N(R) is 
not within range, the N(R) analyzer process N(R)A causes the link to 
be reinitialized (the FRMR primitive is sent), and all previously 
transmitted but unacknowledged I frames are cleared from the packet 
buffer (they remain unacknowledged and are not retransmitted). (Ac- 
cording to OSN 2.3.3.5, detection and recovery from the possible loss 
of such frames is left to a higher-level protocol. I find no reason not 
to provide for recovery in level 2. This could be accomplished merely 
by retransmitting all frames starting with the first unacknowledged 
one, without reinitializing the link. However, the specification given 
below is consistent with OSN 2.3.3.5.) 
states: OK (within range) 

TEST 

FRMR (not within range) 
OK (within range) —> 
TEST: .-(S1’:1) 
OK: _ otherwise 
TEST > 
OK: _ -[(STS:{5,6,7}) + (CP’:{I,RR,RNR,REJ}) OSN 2.4.7c 

+ : Y (UNACK:1)(N(R):n)(V(S):7)] 
i<ngj 
FRMR: otherwise 
FRMR — 
OK: -(STS:3) 
FRMR: otherwise 
SD (2.3.6 sender) 

Classifies current status of header assembler as “to be sent” or 
“already sent.” (This is not an explicit process of OSN; it is used in 
conjunction with the transmitter process TR to coordinate buffering 
of outgoing frame.) 


states: L (last) Header already sent. 

L’ Placekeeping state. 

C (continuing) Header to be sent. 
selections: (SD:L) 
(SD:C) 
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L (last) > 

L’:-[(N(R)A:TEST) + a(aps)(CP:1)] 

L: otherwise 

L’ — 

C: (SD:L)(N(R)A:TEST)[a(aps) + (CP:1)] 
L’: (SD:L)[(N(R)A:TEST) + a(aps)(CP:D)] . 
C (continuing) > 

L:-(TR:ON)(SW:0) 

C: otherwise 


SW (2.3.7 TR shadow) 
Marks the instant that the transmitter process TR turns on and 
off. (Implicit process for control of certain counter.) 
states: 0 (transmitter was OFF) 
1 (transmitter was not OFF) 
0 (transmitter was OFF) — 
1: - (TR:OFF) 
0: otherwise 
1 (transmitter was not OFF) > 
0: - (TR:OFF) 
1: - otherwise 


I (3.1.1 packet counter) 
Indicates the smallest in-sequence level 3-assigned number of a 
packet unacknowledged by the remote end. 
states: N (parameter) N21 
N—- 
N + 1:-(N(R)A:OK)a(nn) 
N: otherwise 


Q (3.1.2 packet buffer queue length) 

Indicates the number of packets N in the packet buffer. (By as- 
sumption, the packets in the packet buffer bear numbers M, M + 1, 
..., M+ N — 1 where M is the state of the packet counter I.) The 
buffer may be loaded by local level 3; this is modelled by a nondeter- 
ministic jump in state from N to N + 1. 


states: N (parameter) 0O< NS Qmax 

N— 

N+1:- . N < Qmax 

M:  - (N(R)A:OK) } 0<M<N 

Li-i]=N-M 

(N(R):i)(UNACK:;}) 

N: - (N(R)A:OK) 

N- 

N+1:- if0 < N< Qmax 
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M+1:- (N(R)A:OK)  } f0O<M<N-1 


Ui-i=N-M 
(N(R):1)(UNACK:;}) 
M: - (N(R)A:OK) } if0<M<N 
i-i]=N-M 
(N(R):i)(UNACK;j) 
N: otherwise 0< NS Qmax 


The transition from N to M + 1 models the 
simultaneous acknowledgment of N — M 
frames and the reception of a frame from 
level 3. ; 


R (3.2.1 packet receiver) 
Indicates the level 3 packet number of last packet passed by level 2 
to level 3. (The passing of the actual packets is modelled here by the 
passing of these packet numbers. The transmission of bits from level 
1 to level 3 is assumed to occur in real time. That is, it is assumed 
that level 2 can transmit to level 3 as fast as level 1 can transmit to 
level 2. The sequence of events during the course of which a packet 
passes from level 1 [through level 2] to level 3 is as follows. An 
incoming information frame is first analyzed by peripheral level 2 
equipment (B). If the frame is not immediately discarded on the basis 
of an invalid opening flag on syntactically bad address/control fields, 
the control field is analyzed by the N(S)/N(R) analyzers. The control 
field is followed immediately by the information field [packet]. Two 
possible ways to treat this packet are (1) to immediately transmit it 
to level 3 and then abort this transmission if the level 2 analysis 
indicates the frame must be discarded or (2) to buffer the packet 
during the course of the level 2 analysis. [In option (2), virtually the 
entire frame must be buffered, as the CRC check and closing-flag- 
valid check cannot be made until the entire frame has been received. ] 
In either case, the checks on the frame are not all simultaneous. When 
the final check test is passed [valid closing flag] the UNACK and 
V(R) counters are updated.) 
states: N (parameter) N2O 
N= 
M .(CP’:I)(N(S)A:OK)(N(R)A:OK)(LS:NB)(I’:M) 


N: otherwise me 


7.5 Level 1 processes 


The eight processes comprising incoming level 1 are modelled here. 
Each process has an acronym which ends with a prime (’). The 
processes of outgoing level 1 are identical but that their acronyms 
omit the prime (’). 
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C/R’ (4.1 incoming L1 address) 
Signals the address field of a frame arriving from remote level 2. 

states: C 

R 

¢ (null state) 
v (parameter) > . v € {C,R} 
o:-(S1':1) 
v: otherwise 
¢ (null state) v € {C,R} 
v -(R1’:1)(TRr:ON)(C/Rr:v) 
g: otherwise 


CP1’ (incoming control primitive) 
Signals the control primitive in the control field of a frame arriving 
from remote level 2. 
states: I For names of states, see control primitive as- 
RR sembler process 1.1.1.2. 


FRMR 

¢ (null state) 
v (parameter) —> v € {I,.... FRMR} 
o:-(S1':1) 
v: otherwise 
¢ (null state) > 
v:-(R1’:1)(TRr:ON)(CPr:v) uv € {I,..., FRMR} 
¢: otherwise 


N(S)1’ (4.3 incoming L1 sequence number) 

Signals the sequence number in the control field of a frame arriving 
from remote level 2. (The state of N(S) is equal to the state of V(S) 
when the frame was transmitted). 
states: N (parameter) 0<N<8 

@ (null state) . 
N (parameter) — 
@ -(S1’:1) 
N: otherwise 
¢ (null state) > 
N-(R1’:1)(TRr:ON)(V(S)r:N)(CPr:1) 
@: otherwise 
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P/F 1’ (4.4 incoming L1 poll/final bit) 
Signals the poll/final bit in the control field of a frame arriving from 

remote level 2. 
states: 0 

1 

@ (null state) 
v (parameter) > v € {0,1} 
@ -(S1':1) 
v: otherwise | 
¢ (null state) > 
v:-(R1’:1)(TRr:ON)(P/Fr:v) 
o: otherwise 

N(R)1’ (4.5 incoming L1 receiver sequence number) 

Signals the state of the received sequence number assembler V(R) 
in the control field of a frame arriving from remote level 2. 
states: N (parameter) 0<N<8 

@ (null state) 
N (parameter) —> 
¢:-(S1':1) 
N: other 
¢ (null state) 
N:-(R1’:1)(TRr:ON)(V(R)r:N)(CPr:{I,RR,RNR,REJ}) 
@: otherwise 

I1’ (4.6 incoming L1 information field) 

Signals the packet number of the packet in the information field of 
a frame arriving from remote level 2. 
states: N (parameter) 0<N 

@ (null state) 
N (parameter) —> 
o:-(S1’:1) 
N: other 
¢ (null state) > 
N:-(R1’:1)(TRr:ON) (Ir:N)(CPr:1) 
@: other 

R1’ (4.7 incoming L1 receive moderator) 

Indicates to remote level 2 whether or not incoming level 1 is ready 
to receive a frame. (This process provides synchronization for trans- 
missions from remote end to local end.) 
states: 0 (not ready to receive) 

1 (ready to receive) 
0 (not ready to receive) — 
1:-(CP1’:¢) 
0:- 
1 (ready to receive) — 
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0:-(TRr:OFF)(S Wr:1) 
1: otherwise 


S1’ (4.8 incoming L1 send moderator) 

Indicates to local level 2 whether or not level 1 is ready to send a 
frame. 
states: 0 (not ready to send) 

1 (ready to receive) 

0 (not ready to receive) — 
1:-(CP1:¢) 
0:- 
1 (ready to receive) — 
0:- 
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Clocked schedules represent an important method of task scheduling for 
computer systems with real-time applications. In this paper we consider a 
generalized class of clocked schedule that includes those used in many stored 
program control switching systems. Key performance measures for this class 
are discussed, and an analytic approximation method for analyzing certain of 
these measures is given. This approximation method is most applicable in 
evaluating long-term delays. (A companion paper by Doshi addresses short- 
term delays for systems with extremely time-critical tasks.) Comparisons are 
made with exact numerical results (obtained using the method presented in a 
companion paper by Ackroyd), detailed simulation models, and field data. 


I. INTRODUCTION 


Processor scheduling concerns specifying when each task that must 
be done in a computer system is to be scheduled for execution, and 
how conflicts in task execution are to be resolved—e.g., by setting task 
priorities. In this paper we consider the performance analysis of a 
class of schedules for computer systems with real-time applications, 
that is, applications where at least some tasks have time-critical 
execution requirements. Collection of digits in a call-processing system 
is one of the more important examples of a time-critical task. 

To introduce the concept of a clocked schedule, we consider the 
simple example system of Fig. 1. The processor must respond in a 
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Fig. 1—Example system. 


timely manner to requests from sources to send data. This consists of 
recognizing new requests, doing setup work (e.g., assigning buffers, 
sending a “ready to receive” signal), and collecting the data sent. For 
concreteness, consider this to be a microprocessor system handling 
incoming dial pulse trunks. Collecting the data corresponds to count- 
ing the number of pulses sent (number dialed). We can distinguish at 
least two distinct tasks: task 1, collecting pulses on active trunks; task 
2, scanning inactive trunks for new requests and performing needed 
setup work. Task 1 is clearly the most time critical. Figure 2 shows a 
clocked schedule for this system. Time is divided into intervals (slots) 
of length T (dictated by the on-off lengths of the pulses), and in each 
slot, first task 1 is executed (for all active trunks), then task 2. If, at 
the beginning of a time slot, task 2 is executing, it is interrupted and 
task 1 execution is given control. 

In applications of this type, the large number of stimuli (e.g., state 
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Fig. 2—Clocked schedule for example system. 
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Fig. 3—Comparison of clocked and interrupt-driven schedule. 


changes on active trunks) that need processing per unit time usually 
precludes the use of an interrupt-driven schedule having one interrupt 
per stimulus. Figure 3 gives a qualitative description of the relative 
trade-offs between clocked and interrupt-driven schedules. For a more 
detailed, quantitative treatment of this trade-off, see, for example, 
Ref. 1. For a discussion of performance trade-offs of other scheduling 
alternatives, see Ref. 2, where this example system is used to provide 
a tutorial on the analysis and design of processor schedules for com- 
puter systems with real-time applications. 

Considerable work on analyzing clocked schedules similar to the 
basic one described above, including applications to distributed micro- 
processor-based systems, has been done by P. Kuehn and others (e.g., 
see Refs. 3, 4, and 5). Here we consider a generalized clocked schedule 
that forms the basis for the processor scheduling in many real-time 
systems, including microprocessor-based systems as well as large sin- 
gle- and multiprocessor call-processing systems. In Section II we define 
the class of schedules to be studied, discuss relevant performance 
measures and work-load characterizations, and use concrete examples 
to illustrate some scheduling variants to this class of schedules. In 
Section III we develop approximations for certain performance meas- 
ures introduced in Section IJ. The approximation method is based on 
that given in Ref. 6. In Section IV we illustrate the accuracy of the 
approximation by using comparisons with exact calculations (using 
the methods described in Ref. 7), simulation, and field data. Section 
V contains some concluding remarks. 


Il. A GENERAL CLASS OF CLOCKED SCHEDULES 


The simple clocked schedule noted above can be generalized in a 
variety of ways. We will consider some of the most important gener- 
alizations from an applications point of view and also note some 
additional variants of practical importance. 
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2.1 Generalized clocked schedule 


Figure 4 shows the general class of clocked schedules that we will 
consider. Time is divided into intervals (slots) of length T. The first 
(labeling) column lists, in order of descending priority, all potential 
tasks for scheduling. A 1 in the ith row and jth column indicates that 
the ith listed task is scheduled for execution in the jth slot. We assume 
that every task is scheduled periodically and that the total period of 
the schedule is Nperioa. Three classes of tasks are indicated on Figure 
(4): High Priority (HP), Low Priority (LP), and Fill (F). The distinc- 
tion between these will become clear in the following discussion. 

The execution of the schedule proceeds as follows. We begin at time 
t = 0 at the beginning of the first slot. The tasks scheduled are 
executed in the order that they appear. Assuming all HP and LP tasks 
are completed before the end of the time slot (¢< T), F work (assumed 
scheduled in every slot) is begun. If an LP task is still executing at 
the end of the time slot, it is interrupted, and it and all lower-priority 
LP tasks are added to the work list of the next slot at their same 
priorities. The HP tasks are not interrupted at the end of the time 
slot, that is, all HP tasks scheduled are completed before new work is 
scheduled. We will thus often refer to HP tasks as noninterruptable 
tasks and LP tasks as interruptable tasks. (F work is also interrupta- 
ble.) . 

Thus, in summary, we have the following execution pattern. When 





TIME ——> 


Fig. 4—General clocked schedule. 


600 TECHNICAL JOURNAL, FEBRUARY 1985 


the HP list of tasks for the jth slot is begun (possibly delayed due to 
past scheduled HP work), the interrupt timer is inhibited. At the 
completion of all HP tasks, the timer is enabled. If the current time 
slot has already expired, the interrupt timer will immediately fire 
(scheduling new HP tasks); otherwise the LP work is begun. If an 
interrupt occurs prior to completion of all LP tasks, the remaining 
work is added to the next slot (the remaining rows in the jth column 
are “or’ed” to the (j + 1)st column). If all LP work completes prior to 
the interrupt, F work is executed until the interrupt occurs. 


2.2 Performance measures 


For a given task, scheduled every 7 time units, we distinguish three 
categories of performance measures: short-term delays—delays on the 
order T or less; medium-term delays—delays greater than T but less 
than 7; and long-term delays—delays greater than +. While this 
categorization is somewhat arbitrary, it has been useful in practice. 

HP tasks are generally reserved for executing the most time-critical 
functions (e.g., digit reception in our example), and hence short-term 
delays are the most important performance measures. Indeed, short- 
term delays generally are needed to determine an appropriate time- 
slot length, T. A particularly important performance measure in this 
category is the probability that HP work scheduled for a given time 
slot is still executing at the end of that slot. This measure is generally 
referred to as the probability of (slot) overrun. Medium- and long- 
term delays also are often important for HP tasks; the latter is often 
needed to set appropriate levels for sanity timers. (In most systems, 
timers are set to monitor the time between successive executions of 
each task. If this time exceeds a given threshold [set to minimize the 
probability of false alarms due to work-load variability], certain main- 
tenance routines are invoked.) 

Long-term and sometimes medium-term delays are usually the most 
important performance measures for LP tasks, which generally are 
less time critical. For example, receiver attachment is typically sched- 
uled as an LP task, possible every 100 ms, but with delays up to 500 
ms acceptable. 

Typical F tasks are line scanning for new originations or mainte- 
nance. For such tasks one defines a quantity T; (e.g., time to scan all 
lines once and time to complete a set of maintenance tasks) such that 
the relevant performance measures are based on delays in completing 
T; time units of F work. (Sometimes F work consists of executing a 
new schedule, in which case distributional information for delays can 
be an input to another performance analysis.) The delays of interest 
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for F work are generally long term, much greater than T, and often 
greater than Nperioa T', the entire schedule period. 


2.3 Work-load characterization 


Two main aspects of work-load characterization must be considered: 
the job-processing times associated with task execution, which gener- 
ally depends on the number of jobs (stimuli) to be processed; and the 
characteristics of the arrival of jobs (stimuli). If we denote by X; the 
(random) amount of time required to process task i, then often X; is 
adequately characterized by a; + b;J;, where a; is the overhead associ- 
ated with entering task 1, J; is the (random) number of jobs found (in 
our example, the number of trunks that had state changes to be 
processed), and b; is the time needed to process each job found. To 
discuss the characterization of the arrival process, we use the queueing 
model description of a clocked schedule given in Fig. 5. There are two 
queues for each HP or LP task, an internal queue and an external 
queue. There is also a “never empty” single queue for F work. The 
arrival rate of jobs for the ith task is denoted by \;—often (we note 
some important exceptions shortly) the assumption of independent 
Poisson streams for these external arrivals is adequate. These external 
arrivals enter the external holding queue and are only allowed to pass 
to the internal queues, where they can be processed, when the indicated 
gates are opened. We distinguish four main arrival characterizations 
based on the gate-closing mechanism. 

Gating 1: Here, at the start of each new time-slot execution, all 
gates corresponding to tasks scheduled for execution in this time slot 
are instantaneously opened, moving all existing jobs from the holding 
queues to the internal queues. The gates are then immediately closed, 
barring further movement of new jobs into the internal queues. 

Gating 2: At the start of execution of a new slot, the gate for the 
highest-priority scheduled task is opened instantaneously and then 
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Fig. 5—Queueing model of clocked schedule. 
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closed. As each new scheduled task is ready for execution, its gate is 
correspondingly opened instantaneously and then closed. 

Gating 3: Same as gating 2, but once opened, each gate is left opened 
until there are no more jobs for this task; the gates are then closed 
and the gate for the next task is opened. 

Gating 4: All gates are always open. 

Although in any given case, one of these gating mechanisms can 
often be used to characterize the arrivals, in actual systems, quite 
often a mixture of all are present, and indeed, there are further 
complications, e.g., the A; are not independent. (Even in our example, 
clearly a large number of state changes in a given slot implies a high 
probability of a large number of state changes in a slot that is displaced 
in time by a dial pulse length.) We will note some examples as we 
discuss some variants of the class of clocked schedules we have defined 
for study. 


2.4 Some variants and examples 


We briefly note some examples of systems that use clocked schedules 
as a means of demonstrating some of the complexities of clocked 
schedules for real systems and how they often deviate from the “ideal” 
models noted above. However, practical experience has indicated that 
these somewhat idealized models can often provide valuable insight 
into system performance. 


2.5 Microprocessor peripheral interface system 


An example of a rather basic clocked schedule is that used in the 
Microprocessor Peripheral Interface System (MPIS), designed to pro- 
vide an intelligent interface between certain switching systems and T- 
carrier facilities. The main purpose of MPIS is to execute the tasks 
associated with the setup and tearing down of interoffice calls via a 
T-carrier facility. This includes digit reception/transmission/timing 
and interoffice signaling. For this system, the schedule for each slot is 
identical. After some overhead associated with each new time slot, the 
first task executed is to poll each of the T-carrier digroups and to 
process signaling change reports from or orders to them. With this 
polling mechanism, some work arriving after the task is initiated will 
be processed, while some will not. Hence the arrivals to this task 
behave like a combination of gating 2 and gating 3. The next task 
consists of unloading a First-In First-Out (FIFO) queue with orders 
from the switching machine and, unless the order is a high-priority 
maintenance order, sorting and scheduling the orders for processing 
by other tasks later in the slot. Gating 1 is a reasonable approximation 
for characterizing the arrival pattern to these latter tasks. However, 
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there is clearly a dependence between the work associated with these 
tasks and the work done in the FIFO task. 


2.6 Support processor 


Large 1 ESS™ switching equipment offices have a Support Processor 
(SP), which performs most of the needed input/output and timing 
functions. Almost all of the tasks are executed as HP, LP, or F tasks 
consistent with the model of Fig. 4. Typical HP tasks include digit 
reception and transmission, LP tasks include timing functions, and 
both trunk and line scanning for new originations are done as F work. 
However, one low-priority (interruptable) task, Peripheral Order 
Buffer (POB) execution, is clearly at variance with Fig. 4. This task 
executes orders that control peripheral equipment, e.g., open (close) 
relays; thus, after completing execution, it must wait a minimum 
period of time (20 ms) for the controllers to free and reset. Hence, this 
task is executed asynchronously with the rest of the schedule being 
rescheduled precisely 20 ms after completing. Moreover, these orders 
can be loaded by the main processor at any time; hence gating 4 
applies. 


2.7 Private branch exchange 


Clocked schedules are common in many Private Branch Exchanges 
(PBXs). Again, HP work usually involves digit reception and trans- 
mission, and LP work includes scanning for new originations and 
receiver attachment, while F work is devoted to maintenance tasks. 

These variants, and many others in actual systems, cause the models 
discussed above to be approximate at best. Nonetheless, as noted, they 
can often supply useful performance information. We will use these 
examples to look at some concrete quantification in Section IV. 


Ill. APPROXIMATE ANALYSIS OF LONG-TERM DELAYS 


For systems where short-term delays are an important performance 
measure, it is usually necessary to capture a considerable amount of 
scheduling detail (e.g., the dependencies and gating noted for MPIS 
in Section II). This problem has been addressed in Ref. 8, where exact 
solutions for realistic models are given. Here we concern ourselves 
with obtaining rather simple analytic approximations to long-term 
delays. In practice, the results have also been useful for medium- and 
sometimes short-term delay calculations. 


3.1 Approximating task waiting time for the simplest system 


Consider the simple clocked schedule depicted in Fig. 6. We assume 
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Fig. 6—Simplest clocked schedule. 


that the arriving work associated with the ith task, i = 1, 2, forms 
sequences {X*}, {X} of independent, identically distributed random 
variables and that these sequences are mutually independent. We also 
assume that the work arrivals follow the gating 1 procedures described 
above. Denote the distribution functions for these work items by F;(x), 
ie., Fi(x) = P,{X; Ss x}. Under these assumptions, the delays for task 
1 can be obtained by studying a D/G/1 queue with (deterministic) 
interarrival time equal to T and service-time distribution given by 
F,(x). Reference 8 gives a variety of approximation methods for 
estimating the delays in such a system. In particular, if we denote the 
waiting time by t,, then Ref. 9 shows how to determine suitable 
parameters a), C; such that 


W(x) =l1- Cyie"™* (1) 


is a good approximation to P,{t; < x}. (We discuss some specific 
choices of a;, C; in Section IV.) To study the delays for task 2, we first 
define a random walk, { Y*; y} by 


yr = y* i (T es xe = y* = VAgie k >0 (2) 


N(y) min} Y* <0, given Y°= vt, 
k=1 


where T is the time slot length. 
Now define: 

Ty = time from arrival of an arbitrary batch of task 2 work until its 
processing begins (task 2 waiting time). 

w = work backlog (task 1 and task 2) just before the start of an 
arbitrary time slot. 
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Wn) = P,{ry > nT}. (Complementary task 1 waiting-time distri- 
bution). 

G.(n; y) = P{N(y) > n}. 

H(x) = P,{w s x}. 
We then have that 


WAn) = i G.(n; x)dH(x), n2=1 
W.(0) = 1 — H(0)F,(0). (3) 


That is, if in the random walk (2), y is the backlog seen by an 
arriving task 2 batch of work, then Z* is the amount of time available 
in the kth slot to work off this backlog (or add to it if Z* < 0), Y* is 
the remaining backlog at the end of the kth slot (start of the (k + 1)st 
slot), and the new work for task 2 arriving at the beginning of slot 1 
begins execution in the first slot that ends with Y* < 0. 

The usefulness of (3) depends on our ability to determine reasonable 
approximations for G,(n; x) and H(x). First, we note that the work 
backlog, w, is precisely what would be seen by an arrival to a D/G/1 
queue with service-time distribution equal to the convolution of F; 
and Fy, i.e., with service time X, + X>. Again the approximation 
methods of Ref. 8 allow us to reasonably approximate H(x) by using 
a simple exponential form, H’(x), i.e., with 


H’(x) = W.(x) = 1 — C.e7%. (4) 


The central limit theorem implies that G,(n; x) is asymptotically 
Gaussian, for large n. We thus approximate G,(n; x) by Gi(n; x) 
defined by 


CO 





1 2 
Gi(n; x) = — e > dy, 5 
(n; x) V2Qa S(n-my)/on ” o 
where 
ip 
NS | 
xoz 
oy = os 
and, as above, Z = T — Xj. 
Equation (5) can be written as 
Guia Ain 
Ge(n; x) = 5 F Erf ae Ao || (6) 
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where 
A, = Z3/N 20% 
Ag = Z?/N 203 


and 


9 Vv 
Erf(v) = Ge i, eds. 
Using (4) and (6) in (3) yields the approximation Wi(n) to W,(n) 


Wi(n) = A3exp(—A,4n), n2=1 


W.(0) = 1 — (1 — C;)F,(0), (7) 
where 


asl 2 


, i ———— 
‘ 2VAz + a[ VA$ + a.—- Ag] 
Ag = 2A;(VA2 + a2 - Ao). 


Equation (7) thus provides our desired approximation of the delay 
distribution for task 2. Note that (7) can readily be evaluated for n 
nonintegral yielding an interpolation formula. 

While the use of (5) might seem to imply that n would need to be 
reasonably large for (7) to be useful, in fact, practical experience has 
indicated that (7) can provide a useful approximation, even for small 
n (e.g., n = 1). This will be illustrated via some examples in Section 
IV. We will look at how other performance measures can be approxi- 
mated, e.g., task sojourn time and delays for F work. First, we look at 
how we can use this method to obtain approximate delays for the more 
general clocked schedule of Fig. 4. 


3.2 Approximating low-priority task waiting time for a general clocked 
schedule 


To treat the generalized clock schedule introduced in Section II, we 
need to extend the above approximation method to include an arbitrary 
number of tasks in a given time slot and, more important, to include 
more complex schedules. The former can be accounted for readily in 
the following manner. Assume for now that all slots have an identical 
schedule and that LP; is the task to be analyzed. We then replace all 
tasks of higher priority than LP; by a single task with an equivalent 
work load, i.e., we make the transformation 

j-l 


Y HP; + } LP; > task 1 
i=1 


1=1 


LP; — task 2 
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and use the above analysis. Two methods of obtaining the needed 
aggregated work-load characterization for task 1 have been found 
useful. The simplest method is to match the relevant means and 
variances. For example, if the work loads take the form a; + 0;<J; for 
the ith task, and ); is assumed Poisson, then we assume that the work 
load for the aggregated task, task 1, has the form a + b.J, where 


a= >» a; 
bJ = Y: bid; 
DJ = ¥. bd; 


Another alternative that is often preferable (particularly if the 
dominant work load is the result of a small number of jobs with large 
processing times) is to match the mean work load and the probability 
that there is no time available in a slot to execute task 2 work. That 
is, in addition to a = ¥: a;, bJ = ¥ bjJ, we must require that 


P,{a + bJ > T} = PAY a; + ¥ bd; > T} = P*. 


The methods of Ref. 8 can be used to obtain P*. A simpler method for 
estimating P* that has proved useful is to use a “clustering” technique 
to replace the potentially large number of tasks that must be aggre- 
gated by precisely three, i.e., 


> a; + > bid; — a, a bids a bjod,2, 


where the > a; and tasks with small values of b; map into a,,, tasks 
with medium values of b; map into b,;J,;, and tasks with large values 
of b; map into b,2J,2. The resulting estimate of P* can then readily be 
computed. This also provides a simple approximation technique for 
the probability of overrun, another important performance measure 
noted earlier. 

We now consider a more complex schedule. Let Nperioa, be the 
scheduling period for LP;. We assume that Nperioa,j divides Nperioa, the 
period of the total schedule. We introduce a new clocked schedule with 
slot length T; = Nperioa,j T, and a task 1 that is a suitable average of 
all tasks of higher priority than task j in all slots, i.e., let Kj = Nperioa/ 
Nperioa,j. and let Iyp,, and Iyp,, be the collection of indices for tasks of 
higher priority than task j in the kth scheduling interval, k = 1,... , 
k;. Then 


Average { ¥ HP; + ¥ wei — task 1 


k=1,..,K;  lichyp.. icIP,k 


LP; — task 2, 
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and we again use the above analysis. (Aggregation methods similar to 
those discussed above can be used here.) 


3.3 Other performance measures 


The method outlined above can be extended to obtain useful ap- 
proximations to other performance measures. We first look at the 
conditional sojourn time of a task. For simplicity of notation, we 
consider the simple system of Fig. 6. The work-load aggregation given 
above can also be used here for more complex schedules. 

We now define: 

ts(y) = time from arrival of an arbitrary batch of task 2 work until 
completion of y units of work on that batch. 

S.(n; y) = P,{rs(y) > nT} (Complementary conditional sojourn 
time). 

We then have that 


S.(n; y) = f Gn; x + y)dH(x). (8) 


Thus if y = 0, then S,(n; 0) = W.(n). 

Using (4) and (6) in (8), we obtain a rather complex integral 
involving the error function Erf(-) that does not seem to have a closed 
form representation in terms of Erf(-). Since our objective is to obtain, 
where possible, simple analytic approximations (exact numerical 
methods are available in Ref. 7), in addition to the Gaussian assump- 
tions above, we assume the following (approximate) decomposition, 
S2(n; y), of S.(n; y): 











Se(n; y) = } W.(n — m)dG’(m; y), (9) 
where 
llx—WN. : 
eel 
(m; y) J2xon, exp | | oN, 
and 
G ¥ 
N,=% 
2 
oh, = Fe 


i.e, G’(m; y) is a (continuous in m) Gaussian approximation to 
P,{N(y) = m}. This results in the following expression for S;(n, y): 
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| 2 V2 o Ny 26 Ny 
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+-—|1—Erf || 10 
2 | | V20n a 
and the A,’s are as before. 

While (10) may seem complicated, it involves nothing more complex 
than an error function, Erf(-), whose efficient computation is available 
in most computer system libraries. 

Equation (10) can be used to obtain approximations for a variety of 


important performance measures. For example, the time from arrival 
to completion of a batch of task 2 work is given by 





sun) = | Se(n; y)dF2(y). (11) 


For the common case where X2 is composed of individual jobs, we can 
use (10) to compute the average sojourn time seen by a job. Also, if, 
as a fill task, we schedule, say, T; of maintenance as the next fill job 
starting at some time point, then S<(n; T;) provides the delay in 
finishing this task (with task 1 including all scheduled [nonfill] tasks). 

Often, F work is repetitive, e.g., continuously scanning a set of lines. 
In such a case, there is no “waiting time” for fill work, i.e., by its 
definition, it starts immediately after completing. In this case, the 
elapsed time until all fill work is done can be approximated simply by 
Gi(n; T's), where T's is the time to scan all lines. 

Finally, we note that in the gating model “gating 1,” jobs are 
additionally delayed as they wait in the outer queue. Since this delay 
is independent of the delays internal to this system, performance 
measures including this delay can be computed by simply convolving 
this external delay with the internal job delays. 


IV. EXAMPLES AND VALIDATION 


The approximation method we have introduced is relatively simple, 
considering the complexity of the problem. Its usefulness, of course, 
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depends on how well it can capture the essence of the delay character- 
istics. We discuss here some applications and comparisons with: (1) 
exact results for the general class of clocked schedules introduced in 
Section II (using gating 1), (2) detailed simulation results that include 
all of the inherent work-load dependencies in an actual system, and 
(3) a comparison with some actual field data. 

The approximation results were computed using a prototype soft- 
ware package, PACS (Performance Analysis of Clocked Schedules). 
This explicit implementation of our approximations determines C; and 
a; in the following manner. First, the service-time distribution in a 
D/G/1 queue is replaced by an appropriate hyperexponential, expo- 
nential, Erlangian, deterministic distribution that matches the first 
two moments of the service time. Then the approximation referred to 
as the A* approximation in Ref. 9 is used to determine C; and a,. (For 
extreme light or heavy loads the approximations A,r, Axo of Ref. 9 are 
used.) 

The first two examples considered use a mean and variance match 
of the aggregated work loads to a Gaussian distribution (see below). 
The third example used the clustering method noted above, while the 
fourth example again used a mean and variance match. 

Example 1: As a first example we consider the simple schedule of 
Fig. 6 with the following parameter values: 


T (Time slot length) = 10 ms 

\; (Poisson arrival rate of jobs for task i) = 0.8/10 ms = 0.08/ms 
b, (Time to process each job for task 1) = 5.0 ms 

be (Time to process each job for task 2) = 4.9 ms. 


The resulting approximation for the backlog (eq. [4]) is 


W.(x) = 1 — Cpe7* = 1 — 0.554e°°™ the variable Z = T — X; = 

10 ms — 5.0 ms (J), where J; = number of jobs found for task 1. 
For eqs. (5) and (10) we thus have 

Z =6.0 ms 

o% = 20.0 ms’. 


The resulting waiting time (eq. [5]) and sojourn time (eq. [10]) for 
task 2 is shown in Fig. 7, where it is compared with the exact results 
obtained by the methods described in Ref. 7. Even for this simple 
example, the exact equilibrium solution is quite complex, being quasi- 
periodic in nature. Our simple method is seen to provide reasonable 
approximations to the waiting and sojourn time. 

Example 2: Here we consider the more complex schedule shown in 
Fig. 8 with the parameters indicated. We look at the sojourn-time 
distribution for task 7. The work-load aggregation discussed above 
thus leads to K = 2, T,; = 40 ms. Task 1 work is characterized by the 
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Fig. 7—Waiting and sojourn times for Example 1. 





TIME IN MILLISECONDS 


Fig. 8—Schedule for Example 2. 


mean and variance of the task i (i = 1, ... , 6) work falling in each of 
two slots (of length 20 ms.) Figure 9 shows the resulting approximation 
for the sojourn-time distribution again compared with the exact results 
obtained from the methods of Ref. 7. 
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Fig. 9—Sojourn time for Example 2. 
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Fig. 10—Receiver attachment delay for Example 3. 


Example 3: As noted in Section II, one of the LP tasks for the PBX 
discussed is the attachment of a digit receiver for a new line origina- 
tion. Figure 10 shows the delays in receiver attachment as computed 
from our approximation method compared with the results of a de- 
tailed simulation model. This is a call-by-call simulation that includes 
most of the work-load dependencies associated with the actual system. 
We again see quite reasonable agreement. 

Example 4: In the SP noted earlier, line scanning is a fill task. 
Figure 11 shows the line-scan delays predicted by our approximation 
as compared with data collected at a field site. We see that the 
approximation results fall well within the observed band of field data. 
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Fig. 11—Line-scan delay for Example 4. 


V. CONCLUDING REMARKS 


The approximation methods presented here, while quite simple in 
nature, have been used in the performance optimization of many 
systems. Often, these relatively easy-to-compute approximations are 
used in the preliminary design phase to determine several feasible 
candidate schedules, as well as to identify performance issues needing 
closer study. The more precise methods of Refs. 7 and 8 then have 
been used to address these issues. The final one or two potential 
schedules generally are then evaluated using detailed call-by-call sim- 
ulation. In the case of modifications to existing systems, field trials 
are conducted for validation. For new systems, predicted performance 
is verified in laboratory models and early field introduction as needed. 
This has helped bring high-performance, high-capacity systems to 
fruition. 
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Numerical Computation of Delays in Clocked 
Schedules 
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A discrete-time model for clocked schedules is described. The distributions 
of the waiting and sojourn times for tasks in this model can be computed by 
a two-stage process. The first stage involves computing the distribution of the 
work awaiting execution at each time slot in the schedule. The second stage 
yields the required delay distributions. The computations in both stages 
involve discrete convolutions, which can be computed via the fast Fourier 
transform. The availability of an exact method of analysis permits the detailed 
study of schedules. It also enables the accuracy of approximate methods of 
analysis, such as those in a companion paper by Fredericks et al., to be 
determined. The present paper considers just one form of scheduling mecha- 
nism; a companion paper by Doshi considers several other mechanisms. 


I. INTRODUCTION 


Clocked schedules provide a means for organizing and controlling 
the execution of tasks in software for switching systems or other 
systems having strict timing requirements.’” This paper describes a 
method for computing schedules. The method is based on the use of a 
discrete-time model, which makes it possible to compute the required 
distributions exactly, except for the effect of computational error. The 
availability of an exact method of analysis has some advantages, even 
though it can be somewhat expensive in terms of the amount of 
computation needed. An exact method permits the detailed study of 
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the behavior of clocked schedules and the comparison of schedules 
that differ only in small details. In addition, it can be used to check 
the accuracy of more economical, approximate methods of analysis. 

The next section of the paper describes the discrete-time model for 
clocked schedules that allow the methods presented here. Section III 
explains the method for the numerical computation of the required 
delay distributions. Section IV describes some aspects of an imple- 
mentation of the method, and Section V gives some illustrative results 
computed by its use. 


il. CLOCKED SCHEDULE MODEL 


There are two aspects of the model that need to be described. First, 
we need to describe the discipline under which the execution of the 
different software tasks, each having its own priority, is organized and 
controlled. Second, we need to specify the statistical characteristics of 
the execution times of the tasks. 


2.1 Organization and control of task execution 


Figure 1 illustrates how task execution is organized. There are Ntask 
tasks, each with a unique priority. The priorities are numbered from 





ARRIVING DEMANDS 
OF PRIORITY 1 





ARRIVING DEMANDS 


OF PRIORITY Nrask 
TL TT 
PRIORITY PRIORITY PRIORITY 
Ntask 1 


Fig. 1—Schedule table and demands for the execution of tasks waiting in a preempt 
resume head-of-line priority queue. 
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one, top priority, to Niasx, lowest priority. The schedule is specified by 
a table whose elements are ones and zeros. The table has Nperioa 
columns and is considered to be repeated periodically. Each tick of the 
system clock, at t = 0, T, 2T, --- , corresponds to one column of the 
schedule table, and each task corresponds to one row of the table. The 
periods between consecutive ticks are referred to as slots. 

When a clock tick occurs, the system consults the schedule table to 
determine which tasks have become due for execution. The tasks due 
for execution are indicated by the presence of ones in the positions 
corresponding to these tasks and the current clock tick. The system 
instantly places demands for the execution of the newly scheduled 
tasks in a queue. This queue, which is illustrated in Fig. 1, is a head- 
of-line priority queue.® In the queue, demands for task execution are 
grouped in order of task priority, the group of demands for the highest- 
priority task being at the front of the queue. 

All the tasks are considered, in this paper, to be of interruptable 
type,! i.e., if a task of priority i is still being executed when a clock 
. tick occurs, its execution is instantly interrupted, rather than running 
on to completion, through the occurrence of the clock tick. The demand 
for the execution of the task is put back in the queue at the head of 
the group of demands of priority i until no demands of higher priority 
remain to be handled. The execution of the task of priority i is then 
resumed where it was left off, with no overhead (extra execution time) 
involved in the resumption. The queue is of the head-of-line, preempt 
resume type. 


2.2 Task execution times 


We assume that the time required to execute a task, given that it 
has exclusive use of the processor, is an independent, discrete random 
variable. This key assumption makes possible the numerical method 
of solution described in the next section. Task execution times are of 
the form kA, where k is a random integer and where the interval A is 
an integer submultiple of the clock period T/A = Neubav. The execution 
time of each task is characterized by the distribution of the random 
integer k. The method of this paper applies to arbitrary distributions 
for k, which need to be available in numerical form. 

If experimental information about the task execution times is avail- 
able, e.g., as histograms, these data can be used directly to provide the 
numerical input required by the analysis. Alternatively, the numerical 
input required can be provided by evaluating a formula that provides 
a model for the execution times of the tasks. One such model, used in 
Ref. 2, is based on the assumption that the task execution time consists 
of an initial overhead a, followed by n executions of a job, where n is 
Poisson distributed and where each execution of the job requires time 


CLOCKED SCHEDULES—II 619 


b. Provided that a and 0 are integer multiples of A, this model fits the 
form assumed in this paper. Examples of the use of this model are 
given in Section V. 


2.3 Performance measures 


Various measures of the performance of a system controlled by a 
clocked schedule may be of interest. Here we deal with two: the 
distribution of the waiting time of a task and the distribution of its 
sojourn time. The waiting time for a task is the time that elapses from 
the instant it is scheduled until its execution starts. The sojourn time 
is the time from the instant a task is scheduled until its execution is 
completed. 

As mentioned before, the execution of a task may be considered to 
consist of a number of repeated executions of a single job. We do not, 
in this paper, deal with the waiting and sojourn times for the individual 
jobs that, in a batch, constitute the task. Of course, the waiting time 
for a task provides a lower bound on the waiting time for a job within 
the task. Similarly, the sojourn time for a task provides an upper 
bound on the sojourn time for a job. 

In this paper we are concerned with the behavior of systems in 
equilibrium. Nonetheless, the delay distributions will generally be time 
dependent. Consider, for example, a task of low priority that is 
scheduled at several slots. The delays for the low-priority task will 
generally be larger at slots where much work of higher priority is also 
scheduled than at slots where little work of higher priority is scheduled. 

In assessing the performance of a schedule to determine whether it 
meets a specification, the average of the delay distributions of a task 
will usually be of prime interest—the average being computed over all 
slots where the task is scheduled. Sometimes, though, the individual 
delay distributions will be of interest, such as when a schedule is being 
analyzed in detail in order to improve it. In either case, the approach 
described in the next section involves computing the delay distribu- 
tions for a task at each slot where it is scheduled. The average 
distribution is obtained by averaging the individual distributions. 


III. COMPUTATION OF DELAY DISTRIBUTIONS 


As shown in the previous section, a system controlled by a clocked 
schedule can be considered as a nonstationary multiclass priority 
queueing system. The analytical solution of such a system would be 
difficult. However, because of the form assumed for the distributions 
of task execution time, numerical results can readily be obtained. In 
this section we detail the computation of the waiting-time distribution. 
The computation of the sojourn distribution is very similar, so it is 
mentioned only briefly. 
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ALREADY WAITING OF 
PRIORITY ABOVE 


WAITING TIME 
| DOES NOT END 
HERE 


PRIORITY / 


WORK ALREADY NEW WORK AND WORK 
WAITING OF 


i WAITING TIME 
INSTANT WHERE 
TASK OF PRIORITY / 
IS SCHEDULED 


Fig. 2—Relationship between task waiting time and the work waiting in the queue. 


We consider a task of priority 1. At the instant the task is scheduled 
to be executed, the demand for its execution joins the queue, behind 
any previously scheduled demands for execution of the task still 
waiting to be handled. Its waiting time consists of the sum of two 
components, as illustrated in Fig. 2. The two parts of the waiting time 
are as follows: 

1. The time required to clear any backlog of work, also of priority 
i, which was already waiting at the instant the task was scheduled. 
The wait of the task is unaffected by any arrivals of priority i after its 
own, so these do not need to be considered. 

2. The time required to clear the backlog of work, if any, of priority 
higher than i that was already waiting at the instant it has scheduled 
plus the time required to deal with any work, of priority higher than 
1, that is scheduled during its wait. 

We now elaborate on our definition of waiting time. The waiting 
time of a task is considered to have ended when the waiting work 
taking precedence over it becomes zero and then remains zero for a 
finite period. The waiting time is not considered to have ended if the 
waiting higher-priority work merely becomes zero for an instant—an 
event that can have finite probability, with the discrete distributions 
considered here. An example of such an event is illustrated in Fig. 2. 

The situation is quite similar for sojourn times. The sojourn time of 
a task ends when the sum of the work in the queue taking precedence 
over the task plus the work remaining to execute the task itself 
becomes zero. In contrast to our definition of the waiting time, the 
sojourn is considered to have ended if this sum becomes zero for even 
just an instant. 
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The distribution of the waiting time for a task of priority i can be 
computed in two stages. The first stage involves computing the distri- 
bution of the work, of priority 1 and higher, awaiting execution im- 
mediately after the task is scheduled. This work is indicated by the 
time ‘A’ in Fig. 2. The second stage involves computing the distribution 
of the waiting time itself from the results of the first stage. This is 
done by a recursive process in which the Ith stage of the recursion 
yields the probability that the waiting time has duration [A. The two 
stages are now described in more detail. 


3.1 Computing the distribution of the work awaiting execution 


We need to compute the distribution of the work, of priority i and 
higher, awaiting execution just after each clock tick. y,, the work of 
priority 1 and higher awaiting execution at t = nT, can be expressed 
in terms of y,-; and x,, the new work joining the queue at t = nT. The 
equation is 


Yn = (Yn-1 TA T)* a Xn» (1) 


where (-)* = max (0, -). This expresses the fact that the work awaiting 
execution is diminished by T' during a slot, but the waiting work 
cannot become negative. 

The right side of (1) involves the addition of a pair of independent 
random variables. The distribution of the sum of independent variables 
is obtained by convolving their distributions, which leads to a formula 
for the distribution of y, in terms of the distribution of y,-1: 


f(n, 1, k) = [Wananal (a — 1, i, k)] #2 g(n, 1, R). (2) 


Terms used in (2) are defined below: 

f(n, i, k) = the probability that y,, the work in the queue, of priority 
i and higher, at t = nT, would require a total time kA for its execution. 
i, though indicated explicitly, is held constant throughout all compu- 
tations. 

q(n, i, k) = the probability that x,, the new work of priority 1 and 
higher, scheduled for execution at t = nT, would require a total time 
RA for its execution. 

WiNeay = the operation of shifting a distribution Neubay places left 
on the k axis and then sweeping the probability on the negative k axis 
up to the origin.” The shift corresponds to the subtraction of T = 
AN.uvav in (1), and the sweep corresponds to the fact that the waiting 
work can be reduced only to zero. 

*, = the operation of convolution with respect to the discrete variable 
k, e.g., u(k) *, v(R) = YE. u(l)v(k — 1). 

By starting with a known distribution f(0, 1, k), such as the distri- 
bution corresponding to an empty queue, (2) can be evaluated repeat- 
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edly to give the distributions f(n, i, k) for successive values of n. The 
volume of computation needed would normally be quite unreasonable 
if (2) were evaluated directly. However, by using a fast Fourier trans- 
form algorithm,’ we can reduce the amount of computation to reason- 
able proportions. 

The distribution of the work arriving at the queue, q(n, i, k) is, 
due to the periodicity of the schedule table, periodic in n. That is, 
q(n, i, k) = q(n + Nperioa, l, R), for all n. Provided that the average work 
scheduled per clock tick is less than T, the queue will be stable, and 
f(n, J, k) will converge to a periodic steady state. (Repeated evaluation 
of eq. [2] is analogous to a stable time-invariant linear system driven 
by a periodic input. After initial transients have decayed, only the 
periodic response remains.) On convergence, we have obtained the 
periodic steady-state distribution, f(m, i, k), for m = 1, 2, --- , Nperioa, 
i.e., for each slot in the schedule. Except in special cases, convergence 
requires an infinite number of iterations, so an appropriate criterion 
must be used, in practice, to determine when the iterations may be 
stopped. 


3.2 Computation of the waiting-time distribution 


The waiting-time distribution can be computed from the work 
distribution, f(m, i, k), by a recursive process. We require the com- 
putation of a sequence of probabilities w(m, i, k), for k = 0,1, ---, 
where w(m, 1, k) denotes the probability that a task of priority i, 
scheduled at slot m, waits a time period of kA until its execution 
starts. Equivalently, we require the computation of the probability 
that the work in the queue, having precedence over the task, first 
becomes zero at time RA after the task is scheduled (and remains zero 
for at least A). 

We define a sequence of conditional distributions r(m, 1, k, 1), | = 0, 
1, --- , where r(m, 1, k, 1) is the probability that the work in the queue 
having precedence over the task of priority i scheduled at slot m is kA, 
given that the waiting time is /A or more (for ! > 0). For / = 0, r(m, i, 
k, 1) is the unconditional distribution of the waiting work that has 
precedence over the task of interest, at slot m. This work is the sum 
of the work, of priority i and higher, left over from the previous slot, 
plus any new work, of priority higher than i, joining the queue at slot 
m. By convolving the distributions of the two parts of the sum, we 
obtain the required distribution: 


r(m, i, k, 0) = [Wenn fm — 1, i, R)] * q(m,i-1,k). (3) 


The convolution in (3) can be computed, as previously, by the use of 
a fast Fourier transform algorithm. 
The probability that the waiting time is zero is given by the origin 
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point of the distribution computed from (38), i.e., w(m, i, 0) = r(m, i, 
0, 0). 

We now consider the computation of the probability that the waiting 
time is JA, for 1! > 0. r(m, 1, 0, 1) is the probability that the work in the 
queue having precedence over the task of interest becomes zero at lA, 
after the task is scheduled, given that the task’s wait has not yet 
ended. The unconditional probability that this work becomes zero at 
IA, after the task is scheduled at slot m is given by 


l-1 
w(m, 1, l) = { 1+ y w(m, 1, in} r(m, l, 0, 1); (4) 
j=0 


i.e., the required probability value is obtained from the previously 
computed points in the waiting distribution and the / = 0 point on the 
conditional distribution r(m, J, R, l). 

To complete the /th stage of the recursion, the conditional distri- 
bution r(m, j, k, | + 1) must be computed. It is the distribution of 
waiting work having precedence over the task of interest at (J + 1)A4 
after the clock tick where the task was scheduled, given that the wait 
did not end at JA, after the task was scheduled. (We consider lA, 
because the wait is not considered to have ended if the waiting work 
becomes zero at /A_ and then immediately becomes nonzero again due 
to the arrival of new high-priority work.) r(m, j, k, | + 1) is obtained 
by modifying r(m, j, k, l) in three ways, as follows: 

1. The k = 0 point is set to zero and the remaining points are 
normalized. This discounts the case where the waiting time is JA. 

2. The conditional distribution is shifted one place to the left on 
the / axis to account for the reduction of the waiting work during an 
elapse of time A. 

3. The conditional distribution is convolved with the distribution 
of the new work, of priority higher than 1, joining the queue at 
(1+ 1)A after the task of interest was scheduled. If none is scheduled 
to join the queue or if (/ + 1)A is not a clock instant, the conditional 
distribution is left unchanged. 

The new conditional distribution is thus given by 


r(m, l, k, l) ~~ r(m, L; 0, 1)5(R) 


ree ) “i g(h), (6) 


r(m, l, k, l) = | vn ( 


where 5(k) = 1, k = 0; 6(k) = 0, otherwise, and where g(k) is the 
distribution of arriving work having priority higher than j, ie., 


Ae q(m + I/Nperica, t — 1, 2), for | mod Nperica = 0 
6(k), otherwise. 
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The recursive scheme defined by (4) and (5) is started with r(m, 1, 
k, 0), given by (3). 

The normalizing division by 1 — w(m, i, 1) in (5) is followed, at the 
next stage.of the recursion, by a corresponding multiplication by 1 — 
w(m, i, l) in (4). In the actual computations, the divisions in (5) and 
the corresponding multiplications in (4) can be omitted. The computed 
intermediate quantities then lose their interpretation as probabilities, 
of course. 

Evaluation of (6) gives the waiting-time distribution for a task 
scheduled at the mth slot in the schedule. As mentioned before, for a 
task scheduled more than once per period of the schedule, the waiting- 
time distribution will generally vary with m. The computations in- 
volving (4) and (5) can be repeated, for each slot where the task is 
scheduled, to obtain the individual waiting distributions. When the 
average of the distributions is required, this can be obtained by 
summing the individual distributions and normalizing the sum. 


3.3 Sojourn distribution 


The computation of the sojourn distribution is similar to the com- 
putation of the waiting distribution, but it differs in two ways. We are 
concerned with when the waiting work of priority 1 and higher becomes 
zero, rather than the work whose priority is higher than 1. Also, the 
sojourn ends even if this work becomes zero for just an instant. 

To consider work of priority 1 and higher, (3) is replaced by 


r(m, i, ky 0) = [Weuniv f(m — 1, i, b)] *2 g(r, i, R) 
or, equivalently from this equation and (2), 
r(m, i, k, 0) = f(m, i, k). (6) 


Equation (5) is used unchanged. So that we do not disregard cases 
where the waiting work becomes zero for just an instant, we use the 
conditional distribution of the waiting work immediately prior to clock 
instants, rather than immediately after. r(m, i, k, | —1) is the condi- 
tional distribution of work at t = (l — 1) A, after the task is scheduled, 
so r(m, 1, k + 1,1 — 1) is the conditional distribution at /A_. Equation 
(6) is therefore replaced by 


s(m, i, l) = {[ = TL s(m, 1, a} rm; t,1,4 =); 


where s(m, i, !) denotes the sojourn distribution for the task or priority 
i scheduled at slot m. 
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IV. IMPLEMENTATION 


The method given in the previous section has been implemented as 
a computer program in which all the convolutions are computed via 
Fast Fourier Transform (FFT) routines.* The delay distributions for 
a task of priority 1 are computed in four principal steps: 

1. Initialization 

2. Distribution of waiting work, via (2) 

3. Waiting-time distributions, via (3, 4) 

4, Sojourn-time distributions, via (6, 7). 

The major part of the initialization step is the computation of the 
distributiorfs q(m, 1, k) for each slot m in the period of the schedule. 
This is done by convolving the individual task execution time distri- 
butions, as specified by the presence of ones in each column of the 
schedule table. 

Convolutions performed via the FFT are cyclic convolutions of finite 
sequences, rather than the aperiodic convolutions of infinite sequences 
required here.’ In principle, this introduces error, due to wraparound 
of the tails of the computed distributions. However, when a suitably 
large FFT block size is used, the magnitude of the tail extending 
beyond the end of the block is small by comparison with the error due 
to arithmetic rounding (or truncation). The effect of wraparound is 
then negligible, with no appreciable effect on the computed results. 

The iterations in computing the distributions of the waiting work 
are computationally expensive. Of course, if the FFT were not used, 
the computation would be more expensive still—by factors of over one 
hundred, for large block sizes. To minimize the computational expense, 
only as many iterations should be used as necessary for a required 
level of accuracy. In using the method, the following criterion has been 
found useful. The computed cumulative distribution of the waiting 
work at the first slot in the schedule is compared with the correspond- 
ing distribution obtained Nperioa iterations previously. In equilibrium, 
these cumulative distributions would be identical. The iterations are 
stopped when the computed cumulative distributions agree completely, 
in element-by-element comparison, to Npiace decimal places, Nplace 
being specified by the user. Experience shows that this is a reasonably 
satisfactory rule for halting the iterations. Experience with examples 
such as the one mentioned below suggests that the use of this criterion 
usually results in the computed cumulative distributions of waiting 
and sojourn times also having an accuracy of about Npiace decimal 
places in each point. If a large value of Nplace is specified, the desired 
accuracy may be unachievable, due to the effects of arithmetic round- 
ing. 

Testing an implementation of the method presents a difficulty 
because the delay distributions can be calculated manually only for 


626 TECHNICAL JOURNAL, FEBRUARY 1985 





0 10 15 


10 
MILLISECONDS TIME IN MILLISECONDS, t 


(a) 

= 1.0 

—— SOJOURN 

> 

$05 

wi 

=) 

a 0 

0 10 20 30 40 
TIME IN MILLISECONDS, t 

(b) 


Fig. 3—Example of a simple schedule whose delay distributions can readily be 
calculated. 


simple, somewhat atypical examples. Statistical simulations can be 
used to check for gross errors in the implementation, but it would be 
impractical to use simulation results to determine the accuracy of the 
extremes of the distribution tails. One example whose solution can be 
calculated manually is a schedule with a single task, scheduled every 
clock instant. If the execution time of the task is zero with probability 
a, and 2T with probability 1 — a, it is straightforward to show that 
the waiting-time distribution is geometric, with parameter (1 — a)/a. 
In the present form of the program, the FFTs are computed in 
double precision floating-point arithmetic, with the remainder of the 
computation being done in single precision. The accuracy of the 
computed delay distributions depends on the particular problem, but 
results are typically accurate to four decimal places when an IBM 
3081 computer is used. When a VAX* machine is used, which has the 
same word length but has different floating-point arithmetic charac- 
teristics, results are typically accurate to five decimal places. 


V. EXAMPLES 


Figure 3a illustrates a simple example of a clocked schedule; the 
schedule table and the distributions of the execution times for the 


* Trademark of Digital Equipment Corporation. 
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Fig. 4—Example of a schedule where the waiting and sojourn distributions depend 
on the slot at which the task is scheduled. 


individual tasks are shown. The task of higher priority always takes 5 
ms to execute. The task of lower priority, if it had sole use of the 
processor, would take 10 or 15 ms, with equal probability. This example 
is unusual in that the waiting and sojourn distributions can readily be 
computed manually. For the task of lower priority, the waiting time is 
always 5 ms—the time it waits while the higher-priority task is 
executed. In each slot, there are 5 ms available for the task of lower 
priority. Its sojourn time is therefore, with equal probability, 20 or 30 
ms. The complementary cumulative distributions of waiting and so- 
journ time are shown in Fig. 3b. 

Figure 4a illustrates another simple clocked schedule. For each task, 
the execution time has the form a + nb, where a denotes the overhead 
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Fig. 5—Two schedules, identical except for the slot at which task 3 is scheduled, 
illustrating how delays can depend on details of the schedule. 


incurred in starting the execution of the task, b denotes the time to 
execute a single job, and n is the number of jobs in the task, n being 
Poisson distributed. This is a form for the task execution time assumed 
in Ref. 1. Figure 4a shows the schedule table and, for each task, the 
values of a and b, and of 7, the expected number of jobs in the task. 
Figure 4b shows the complementary cumulative distributions of the 
waiting and sojourn times for task 4. In this example, the delay 
distributions differ according to the time slot at which the task is 
scheduled. The individual distributions are shown hatched; their av- 
erage is shown as a continuous line. The presence of cusps is quite 
typical of delay distributions for low-priority tasks. They exist because, 
if a task of low priority is still being executed when a clock tick occurs, 
then it will probably have to wait considerably longer, while the newly 
scheduled high-priority work is executed. 

Figure 5a illustrates two simple schedules; they differ only in the 
slot at which the task of lowest priority is scheduled. Figure 5b shows 
the resulting delay distributions for the task of lowest priority. An 

‘apparently minor change in the schedule results in significant in- 
creases in delay. 
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Vi. CONCLUSION 


A method has been presented, based on the use of a discrete-time 
model, for the numerical computation of delay distributions for inter- 
ruptable tasks in clocked schedules. The method is particularly useful 
when a detailed examination of the characteristics of a schedule is 
needed, such as how the delays depend on the slot where a task is 
scheduled. It does not depend on assumptions that delays are long, so 
it gives information about delays that are smaller than a clock period, 
as well as for longer delays. An important use of the method is checking 
the accuracy of computationally more economical, though approxi- 
mate, methods of analysis. 

Only systems in equilibrium have been discussed here. However, the 
method is capable of being extended to the analysis of transient 
conditions such as those that occur when a system controlled by a 
clocked schedule is subjected to a sudden increase of traffic. 

The method presented here uses an iterative method to compute 
the equilibrium distribution of the work awaiting execution at each 
slot in the schedule. An alternative approach involves the direct 
solution of the equilibrium equations. Levinson’s method for the 
solution of block Toeplitz systems’ has been used, experimentally, for 
the analysis of clocked schedules by the direct solution of the equilib- 
rium equations. However, this approach must be further developed in 
order to become a practical alternative to the iterative method de- 
scribed in this paper. 
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Clocked schedules are used in a variety of real-time systems to perform 
tasks in accordance with their delay requirements. A comprehensive overview 
of various clocked schedules can be found in the accompanying paper by 
Fredericks et al. For one model of a clocked schedule we can see approximate 
and exact analysis of the relevant performance measures in the papers by 
Fredericks et al. and by Ackroyd in this issue of the Journal. These results 
can also be used to evaluate the long-term delays in the other models. For 
extremely time-critical (high-priority) tasks, the probability of tasks not 
getting served in the scheduled interval and the short-term delay distribution 
are important performance measures that are sensitive to the detail structure 
of the clocked schedule. In this paper, we show that these performance 
measures can be obtained in terms of steady-state distribution of an embedded 
Markov chain. This steady-state distribution is calculated, exactly or approx- 
imately, for a number of models, and the results are used to compare, 
numerically, various scheduling mechanisms. 


I. INTRODUCTION 


Clocked schedules are used in a variety of real-time systems to 
perform tasks in accordance with their timing requirements. These 
real-time systems (switching, monitoring, etc.) are characterized by 
having to perform some tasks with extremely stringent timing require- 
ments. We call these high-priority tasks. Besides these, real-time 
systems also perform tasks that can tolerate somewhat longer delays 


* AT&T Bell Laboratories. 
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without affecting performance. These will be called low-priority tasks. 
Finally, these systems perform a variety of background tasks (audits, 
maintenance, etc.), which have very liberal timing requirements but 
which require a specified minimum fraction of the processor time over 
a reasonably long period of time. These tasks are usually performed 
when no high- or low-priority work is present in the system. We call 
these the fill tasks. We emphasize that the boundaries among these 
categories of tasks are not clear-cut and that within each category the 
tasks may have significantly differing time scales. There are various 
mechanisms for allocating the processor time to meet the varying 
requirements of the individual tasks. These mechanisms and their 
relative merits are discussed in Ref. 1. Clocked schedules provide a 
very effective and reliable mechanism to achieve the desired timing 
objectives. Of course, to use them effectively, it is necessary to under- 
stand their performance as it applies to the various categories of tasks. 
Considerable progress has been made in this direction (see Refs. 1, 2, 
and 5 through 8). For low-priority and fill tasks approximate analysis 
of queueing models is described in Ref. 1. Exact numerical procedures 
for these models are developed in Ref. 2. For many systems the 
analyses in Refs. 1 and 2 can also be used to study high-priority tasks. 
However, for other systems the modeling assumptions may not ade- 
quately capture the working of the schedule to accurately analyze the 
short-term delay of high-priority tasks. For such systems it is necessary 
to incorporate additional features into the clocked schedule model and 
understand the implications of these features. Some of these features 
are the gating mechanism, the strategies used under overrun, and the 
dependence introduced by autonomous work generation. In this paper 
we consider increasingly complex models of clocked schedule by intro- 
ducing these features one at a time, and we develop methodology to 
study the performance measures for the high-priority tasks in these 
situations. To analyze these models in a relatively simple way, we 
make assumptions that restrict our analysis only to high-priority tasks. 

This paper is organized as follows: In Section II we briefly define 
the clocked schedule and the performance measures that we will use. 
We also introduce various gating mechanisms and overrun strategies 
in that section. In Section III we analyze a simple clocked schedule 
with respect to the performance measures for the high-priority tasks. 
We also use this analysis to numerically compare various scheduling 
and priority mechanisms. Section IV deals with various generalizations 
and complex clocked schedules. 


I. MODEL OF A CLOCKED SCHEDULE 


The mechanism underlying a clocked schedule has been discussed 
in detail in Ref. 1, so we will be very brief here. 
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Fig. 1—Multiclass queueing structure in clocked schedules. 


Assume that there are m high-priority tasks and n low-priority tasks 
numbered {1, 2,...,m,m+1,...,m-+n}. Each task has two queues, 
the external queue and the internal queue (see Fig. 1). Jobs arrive at 
the external queues and, at specified times, are transferred to the 
corresponding internal queues. The internal queues are served accord- 
ing to a priority scheme, with lower-numbered queues having higher 
priority. The priority of the high-priority queue over the low-priority 
queue and the fill work is preemptive. The priority between two high- 
priority queues is determined by the overrun strategy discussed below. 

The time is divided into intervals of length T ms. During each of 
these time slots some high-priority tasks and some low-priority tasks 
are scheduled (for an example see Fig. 2). The jobs in the queues for 
the tasks scheduled in a given slot are typically moved from the 
external queues to the internal queues in that time interval. The gating 
mechanism determines when this transfer happens. Three gating 
mechanisms are used in practice, the actual choice depending on the 
application and hardware limitations. The first gating mechanism 
(G1) transfers the jobs in the external queues to the internal queues 
for all tasks scheduled in a given interval at the beginning of that 
interval. No more transfer takes place during that interval. In the 
second gating mechanism (G2) the processor starts with task 1 at the 
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1= TASK SCHEDULED 
0 = TASK NOT SCHEDULED 


Fig. 2—An example of a clocked schedule for m = 2, n = 2. 


beginning of each interval. If it is scheduled for that interval the 
transfer takes place and the processor serves the internal queue for 
task 1 to completion. During this period no additional transfer takes 
place for queue 1. After exhausting the first queue the processor moves 
to queue 2. If the corresponding task is scheduled, the transfer takes 
place at this time, and so on. Note that the order of service for the 
internal queues is always (1, 2,..., m+n, fill). The schedule table 
only indicates when the transfer takes place for each task. The third 
gating mechanism (G32) is essentially similar to G2, the difference 
being that when the processor is serving an internal queue, the new 
arrivals to the corresponding external queue move immediately to the 
internal queue. 

If all the internal queues become empty before any interval ends, 
then the processor does fill work until the end of that interval. If the 
interval ends while the processor is working on a low-priority job, then 
that work is preempted and the processor moves to queue 1. If the 
processor is working on a high-priority queue when the interval ends, 
then we say an overrun has occurred. We consider three different 
overrun strategies, S1,S2, and S3. In strategy S1 (Fig. 3) the remaining 
high-priority work in the internal queues is expelled. This is done in 
some systems either because the necessary bookkeeping ability is not 
available or because serving a high-priority task after a reasonably 
long delay is most likely to result in unsuccessful service and waste of 
the processor time. More importantly, as we will see in Section IV, 
this strategy is the starting point of our analysis for more common 
strategies S2 and S3, which are discussed below. The strategy S2 (Fig. 
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INCOMPLETE WORK DESTROYED 


Fig. 3—Strategy S1, in which incomplete work is destroyed. 





WORK RESCHEDULED 
Fig. 4—Strategy S2, in which incomplete work is rescheduled. 


4) under overrun is the same as that used for interrupting a low- 
priority task, namely, the incomplete work is rescheduled and the 
processor moves back to queue 1. In strategy S3 (Fig. 5) the end of 
interval is ignored until all the high-priority internal queues are empty. 
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WORK CARRIED OVER 


Fig. 5—Strategy S3, in which incomplete work is carried over. 


Only then does the processor move to queue 1. Note, however, that 
the end of interval is usually marked by an interrupt from an asyn- 
chronous clock. Thus in $3 an overrun reduces the time available in 
the next interval. 

As mentioned earlier, the high-priority tasks are typically very time 
critical. In this paper we consider high-priority tasks as those which 
we require to be completed in the scheduled interval with very high 
probability. This dictates which performance measures capture our 
interest. 

We are interested in the probability of an overrun. Assuming that a 
high-priority task is scheduled once every p time slots, we are inter- 
ested in the delay distribution in the range [pT, (p + 1)T]. Once we 
address these two questions successfully, we can answer the other 
important questions: What is the largest T for which the delay criteria 
for all high-priority tasks are satisfied? This frequently determines 
the best value of T. Which is the best priority order for the high- 
priority tasks? 

Since we are interested in the delay distribution over a short time 
range, the effects of gating mechanism and overrun may be quite 
significant here. In this paper we develop a methodology to address 
the issues discussed above in a manner suitable for relatively easy 
computations. We begin with a rather simple special case of the general 
model and then introduce complexity in the schedule gradually. 
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Fig. 6—A simple clocked schedule. 


Il. ANALYSIS OF A SIMPLE CLOCKED SCHEDULE 


Recall that high-priority tasks have preemptive priority over low- 
priority and fill tasks and so are not affected by them. Thus we can 
simplify the description of the schedule by replacing all the low- 
priority and the fill tasks with a single task, m + 1. 

The special case we study here (see Fig. 6) is a clocked schedule in 
which each of the m high-priority tasks is scheduled in each interval. 
Thus the entire schedule has a period of one time interval (7). In 
addition we assume that the jobs in the external queue i arrive 
according to a Poisson process at rate \;, and that the arrival processes 
in different queues are independent. When the processor starts serving 
queue 1, it ineurs an overhead OH; with distribution function H;. Each 
job in queue 1 has service time _X; with distribution function F;, 1 = 1, 
...,m. Fori=1, 2,..., m, let 


eel xdH;(x), 
0 


a; = i) o> a;)°dH;(x), 


al xdF; (x), 
0 


Bi = i) xPdFi(2), 
0 


pi = Aibi, 
and 
Yi = AiBi. 
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We analyze this clocked schedule with overrun strategy S1 and 
gating mechanisms G1 and G2. Note that we require the high-priority 
queues to exhaust in every interval with high probability, so the 
difference between overrun strategies S1, S2, and S3 should be small. 


3.1 Analysis for gating mechanism G1 


The analysis for the gating mechanism G1 is simple. First consider 
an arbitrary clock interval of length T. Suppose we let this interval 
continue until all the internal high-priority queues are empty. Let ¢; 
denote the time from the beginning of this interval to the moment the 
processor exhausts internal queue i. sae 


=> OH; + Y y Xie, (1) 
j=l k=1 
where Xjx is the service time of the kth job in the jth queue and N,; is 
the number of jobs served by the processor from the jth queue. Nj; hae 
Poisson distribution with mean ),;T. Thus, if G;, H; and F; denote the 
Laplace transforms of ¢;, OH: and X;, respectively, then 


Gils) = TL [Ay (s)e 2-H], (2) 
a 

The above equation can be inverted to obtain the distribution of ¢;. 
This can be done, for example, by using the transform inversion 
technique discussed in Ref. 3. Alternatively, the convolutions involved 
in eq. (1) can be carried out directly using fast Fourier transforms 
after suitably discretizing the random variables.” Obviously, an over- 
run occurs when t,, > T. Thus, 


Pf{overrun} = P{tn > T} = 1 —- G,,(T). 
Also, 


1 = Gn(s) (3) 
S 


ie e“(1 — G,,(t))dt = 


Thus, we can get P{overrun} by inverting the right-hand side of eq. 
(3) at T. 

Next, we obtain the sojourn time distribution for a job arriving at 
queue i assuming that it does get served in the following time slot. Let 
S; denote the sojourn time for an arbitrary job arriving at queue 1, that 
is, S; is the time between the arrival of a job at queue 7 and the 
completion of its service. Suppose that a job arrives at queue 7 when 
R time units are left to the end of a time slot and that there are N; 
jobs waiting in that external queue at that time. Then R is uniformly 
distributed on (0, T’) and, conditioned on R, N; has Poisson distribution 
with mean );(T — R). The sojourn time, S;, is then given by 
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Nl 


S;= R+ tei + > Xj + OH;. 
jal 


Thus, if S; denotes the Laplace transform of S;, then 
Hii(s)Gi-s(s)Fi(s)[e70-F — eT] 

T(s — d(1 — Fi(s)) 
Once again, eq. (4) can be inverted to obtain the distribution of S;. 


We next obtain the mean and variance of t; and the mean of §;. 
From eq. (2) we easily get 


Si(s) = (4) 


Ele] =D (q + pT), (5) 
and 
Var[t;] = ) (a; + 7;T). (6) 


Also, by differentiating eq. (4) and setting s = 0, we get 
i~1 


S.=ESI= 3 @+ojT)+b+5+p)ta (1 


3.2 Analysis for gating mechanism G2 


We next analyze the simple clocked schedule under gating mecha- 
nism G2. The overrun strategy is still assumed to be S1. Our general 
approach is to define an embedded Markov chain and show that the 
relevant performance measures can be obtained easily in terms of the 
steady-state distribution of this Markov chain. We then find expres- 
sion for this steady-state distribution. 

Let N,,,; denote the number of jobs in the external queue 7 when the 
nth time slot of length T begins. Then {(Nni, Nno, --+ Nam): n 2 1} is 
an ergodic Markov chain. Let (Ni, No, --- N,) denote the steady- 
state random vector corresponding to this Markov chain and let 


P(z1, 22, +++ » 2m) denote the generating function of the random vector 
(NM --- Nm), that is, 
Plas, 225 +++ 5 2m) = Elztizhs «+ 2h], 


We first show that the probability of overrun and the sojourn time 
distribution for any queue can be obtained easily in terms of P. 
Consider an arbitrary time slot and, as before, allow all high-priority 
internal queues to be completely served in this slot by letting the 
available interval overrun, if necessary. Let t; denote the time from 
the beginning of this time slot until the completion of service to the 
internal queue i. Let (Ni, No, --- Nm) denote the number in the 
external queues at the beginning of this slot, and for i = 1, 2, --- m, 


CLOCKED SCHEDULES—II! 641 


let N/ denote the number transferred from the ith external queue to 
the internal queue when the service to that queue begins. Let t, = 0. 
Then for: = 1, 2, ---,m, 


N; 
t; = t-1 + OH; + ¥ Xy, (8) 
j=l 
Nj; — N; + Mi(ti-1), (9) 
where 
M;(x) ~ Poisson (A;x). (10) 
Equations (8) through (10) allow us to calculate the distributions of t; 
and Nj, i = 1, 2, --- m, recursively. In terms of transforms, let 
G(s1, So, +++ Sm) = Eleq“itrtsatet---+8mtm)], 
Then, G can be obtained recursively from eqs. (8) through (10) as 
follows: Let s = (si, S2, --+ , 8m). Define o;(s) and w;(s), i= 1, «++, m, 
recursively by 
Om(S) = Sm) 


@m(S) = Fn (om(8)), 


g;(s) = Kivi(l — wi+1(S)) + gi+i(S) + Si, 1<i<sm-1, (11) 
and 
wi(s) = Fi(o(s)), 1<i<m-—1. 
Then 


G(s) = Plox(s), w2(s), w2(s), «+» om(s)] IT Helos). (12) 


The marginal Laplace transform of t,, now follows by putting s = s‘” 
= (0,0, --- , Sm). Once again, this can be inverted to get the distribution 
function of ¢,. In particular, 


P{overrun} = 1 — P{tn < T}, 
and 


eo G(s) 


{ e"'Pi{tm > t}dt = (13) 
0 


Thus P{overrun} can be obtained by inverting the right-hand side in 
eq. (13) at T. 

We next derive the distribution function of the sojourn time in a 
given queue in terms of P. Suppose we are considering queue i. We 
assume that the probability that the task 1 overruns the interval is 
essentially zero. Let S; denote sojourn time of an arbitrary job in queue 
i and §S; its Laplace transform. Let N% and N% denote, respectively, 
the numbers in the internal queue 1 and the external queue 1 when an 
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arbitrary job in queue i completes service. Let N/= Ni, + Nis. Finally, 
let Nj be the number of jobs in the external queue 1 just before they 
are transferred to the internal queue and the processor starts working 
on them. Let 7;,, 7:2, 7;, and q; denote the generating functions of Ni, 

i2, Ni, and Ni, respectively. We express S; in terms of 7, * in terms 
of qi, and q; in terms of P. Combining these relations we will get S; in 
terms of P. 

First note that, since the service within a given queue is First-In 
First-Out (FIFO), N/denotes the number of arrivals to queue 1 during 
a time interval of length S;. Thus, from Ref. 4 we get 


Si(s) = F(1 — s/X,). (14) 
Next, 
F(z) = E[z%*] 
= E[E[z™*| Ni] 
= E[E(E[2z%**™*2| Ki]| NJ], (15) 


where K; is the position of the job that completed service in the batch 

of size Nj transferred to the ith internal queue when the processor 

last arrived there. From eq. (15) we get 

F(z) = E[E[2N*F,(\(1 — z))“Hj(\(1 — z)) | NA] 
_ FL, (A(1 — 2))(Fi(d — 2))[Gi(z) -— GROW — z)))] 

ni(z — Fi(\(1 — z))) 

where n; = E[N/]. Finally, by the arguments used to derive eq. (12) we 

obtain the following relation between g; and P: Let 


gi-1(z) = A,(1 — 2), 


(16) 


wj-i(z) = F;_,[o;-1(z)], 
oj(z) = AL — wisi(z)) + ofsr(x), Ls).s2— 1, 
and 
wj(z) = Fi(o}(z)) 1<j<i- 2. 
Then 
Gi(z) = Elz” 


i-] 


= I H;(o}(z))P(wi(z), w3(z), +++, wfa(z), z). (17) 


Combining eqs. (14), (16), and (17) we obtain S; in terms of P. 

Thus, all performance measures of interest can be expressed in 
terms of the steady-state distribution of the vector (Ni, No, -:- , Nm). 
We next evaluate this distribution P(k, lb, «++, In) or, equivalently, 
its generating function P(z;, z2, --+ , 2m). Let 
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Q(K, Ka, Peat oy Knlh, L, anaes | lm) 
= P{Nasia _ ky, Nn+12 = ko, ade: Nn+i,m 
= Rm|Nna = h, +++, Nnm = 1n} 


be the transition probability function for the Markov chain {(N,, 
Nn, +++ » Nam): n = 1}. Then P is the unique nonnegative solution of 


P(k,, ke, a Rm) 


= >, by »y P(h, ls, -++, In) Q(Ri, Ro, -++y kmlh, see ydn), (18) 
ln= 


I=0 =0 1,=0 
for k; = 0,1 ,t=1,2---m,and 
y Lees DY Pl +++ kn) = 1. (19) 
ky=0 ko=0 Rm=0 


Q can be readily expressed in terms of 1, lo, --- , Im, Ri, Ro, «+ » Rm and 
the arrival and service time parameters. In particular, it is clear that 
Q(Ri, «++ Rmili, b, «++ Im) does not depend on I,,. Thus, if we define 


P;(Ry, ke, +++, Ri) = PIN, = kh, «++, Ni = Ri} (20) 
and 
Qi(ki --- kl h, In, «++, Uj-1) 
= P{Nn+a = i, +++, Nati = Ril Naa 
=], +++, Nav = b-1}, (21) 


for 1 <i<™m, then 


Pr(hi, saa Rm) 


It 
Ms 
ye 


Seite >: Pri ++: In) 
Im =0 


~ 


0 


*Qm(Ri, +++ mil +++ Im-1) 


1 


o 


Soin Palos) 


4=0 In-1 =0 


Qn (Ri eee Rn|h eee lm—1), (22) 


and, in general, for i = 2, --- m, 
Pi(ki, +++ Ri) = > aa x Pi-lh, +--+, G-1) 
=0 i,_4=0 


Qilki +++, Rill +++, ba). (23) 
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Thus P;, Po, --- , Pm can be calculated recursively using eq. (23). We 
start these recursions with 


et), T)™ 
7 ky = 0, coe 


These recursions use the functions Q; for i = 1, 2, --- m. Of course, 
some form of discretization is necessary to carry out these recursions. 
For large m it is convenient to have a closed form approximation 
instead. Such an approximation is possible if we assume that the 
probability of an overrun occurring by queue 1, i = 1, --- m — 1 is 
negligible. We illustrate this approximation below for the case m = 2 
and then write down the approximation for general m. For m = 2, the 
exact generating function is given by 


P(z1, 22) = E[zt23] 
= E[E[zM282|t{]] 


= E[e™ T(1—21)—Ag T(L~2_)+Aoti(l—z20)) 


P,(k,) = 


where tj represents the time from the beginning of the previous s. ot 
until the completion of all work in the internal queue 1 in that sl: t. 
Thus, 


y e™T(), T) 4 


ey Sy 
2 T(1~22) f 1) dH,(x,)dF\(s, — 21) 
s,=T x;=07 


P(x, 22) — ea TU-21)—AgT(1—29) , 


T Sy ; 
+ i, if dHy(x1)dFY?(s; — x,)e20-*) (24) 
8,;=07 “x ;=0 


Our approximation is equivalent to replacing the first term in the 
squared bracket in eq. (24) by 


00 st 
f ‘| dHy(x;)dF\(s, — ze”), 
5,=T x,=07 


If we denote the resulting approximation for P by P, then 
Pi 21, 22) = H—)o(1 — 29)) eM T(1=2y)—Ag T(=29)—My TUF; (-2g(1-29))) (25) 


How good is P an approximation for P? If we put z, = 1 in eqs. (24) 
and (25) and then invert the resulting generating functions for No, we 
can compare the exact and approximate distributions for No. Tables I 
through III give such distributions for three sets of parameters. The 
approximation seems remarkably good in these cases. These tables 
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Table |—Distribution of the number in 
queue 2 at the beginning of an interval 
with Ay = 0.05,A2 = 0.10,b; = 1.0,b2 = 


1.0, T= 20.0 
P(N2= k) 

Approximation 

k Exact Using P 

0 0.055308 0.055308 

1 0.104504 0.104504 

2 0.099035 0.099035 

3 0.062750 0.062750 

4 0.029901 0.029901 

5 0.003648 0.003648 

6 0.001001 0.001001 

1 0.000241 0.000241 

8 0.000052 0.000052 


Table Il—Distribution of the number in 
queue 2 at the beginning of an interval 
with A, = 0.05,A2 = 0.10,b; = 2.0,b2 = 


1.0,7 = 20.0 
P(N2 =k) 
Approximation 
k Exact Using P 
0 0.062126 0.062126 
1 0.109075 0.109075 
2 0.097270 0.097270 
3 0.058602 0.058602 
4 0.026782 0.026782 
5 0.009889 0.009889 
6 0.003069 0.003069 
7 0.000823 0.000823 
8 0.000194 0.000194 
Table !1|—Distribution of the number 


in queue 2 at the beginning of an 
interval with A; = 0.05,A2 = 0.10,b; = 
3.0,bz = 1.0,T = 20.0 


P(N? = k) 
Approximation 
k Exact Using P 
0 0.070641 0.070636 
1 0.112676 0.112681 
2 0.094152 0.094152 
3 0.054193 0.054193 
4 0.023988 0.023988 
5 0.008665 0.008665 
6 0.002651 0.002651 
7 0.000704 0.000704 
8 0.000166 0.000166 
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also suggest the region in which the approximation is likely to work 
well. 
In general let P,, represent the approximation for P,, and for 1 <i 
m — 1, let 


Piz, 22, °°? zi) = Pozi 22, ° 8% 5 Sis 1 1, ae 1). (26) 
Then we can express P; in terms of P;-, as follows, thus enabling a 
simple recursive calculation of P,: For z = (21, 22, +++ , 2m), let 


gi(z) = A(1 — 2), 
Um(z) = om(z), 
and forl <i<m-—1l, 


Wi(z) = bi(z) + Wisr(z) — 4 — Fi(—isr(z))). (27) 


P,, (2; ae Zin) 


oth ne 


= in H,{- Vi+1(Z)] 
-Pr(Fy(—Wo(z)), Fo(—Walz)), +++» Fm—(—Ym(z)), 1). (28) 


This represents P,, in terms of P,,-;. Applying this equation succes- 
sively m times we get P,, in terms of Pm_1, Pn-2, ---, P; and Po = 
P,,(1, 1, ---, 1) = 1. Thus P,, can be obtained recursively from eq. 
(28). 


3.3 Moments for gating mechanism G2 


We again look at the case m = 2 and obtain approximate expressions 
for the first two moments of t; and te, where t, and t, are as defined in 
Section 3.1. We also obtain the mean values of the sojourn times, S;,, 
i = 1, 2, assuming that the probability of an overrun is negligible. We 
compare these expressions with those for gating mechanism G, and 
also with the gating mechanism G2 but with indices 1 and 2 reversed, 
that is, with queue 2 served at higher priority than queue 1. We will 
use our approximation throughout this analysis. First, from eqs. (12) 
and (28) we get 


G(s, s2) = Ele ~ 2] 
= Ho(s»)Hi(sy + s2 + do(1 — Fo(s2))Hy(—Ao(1 — Fa(s2)))) 
se T[do(1 — Fo(so)) + Aul[2 — Fi(si + 82 
+ do(1 — Fo(s2)) — Fi(—do(1 — Fo(s2))))]1- (29) 
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Putting s. = 0(s; = 0) in eq. (29) we get the approximate Laplace 
transform of t;(t2). Let 


&(s) = Log G(s, 0) (30) 
and 
£(s) = Log G(0, s). (31) 
Then 
E[ti] = — g7(0*) (32) 
and 
var[ti] = 87(0") (33) 


for i = 1, 2. Substituting from eq. (29) into eqs. (30) through (33), we 
get 


E{t;] = a, + pT, (34a) 
E[te] = a1 + ao + (pi + po) T, (34b) 
Var{[t,] =a,+ 1 T, (34c) 


and 
Var[t2] = a1 + a2 + T(yi + Y2) + 2pe(1 + p2)(ar + yiT). (35) 


Comparing these with the expressions derived in Section 3.1 for the 
gating mechanism G1, we conclude that the mean values of t; and tp 
do not depend on the gating mechanism. The variance of t, also 
remains the same. However, the variance of t, for the gating G. is 
larger than that for gating G,. The difference is 


2p2(1 + p2)(ar + yiT) > 0. (36) 


This implies that the probability of an overrun is likely to be larger 
for gating G2 than for gating G1. The difference depends on the 
average load in queue 2 (p2) and the variability of the load in queue 1 
(a, + y:T). Since, in general, it is much easier to analyze G1 than to 
analyze G2, we frequently use the gating G1 as an approximation for 
gating G2. The left-hand side in (36) then gives a measure of this 
approximation. If this quantity is small the approximation is likely to 
be good. We will come back to this point in Section 3.4. 

Next, we see what happens if we interchange queues 1 and 2. In 
other words, the parameters of the arrival and service time for these 
two queues are interchanged and we compare the moments of ¢, and 
t. for these two orders. We will use suffix 0 for the original order and 
R for the reversed order. Then, 
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Ealth] — Eolti] = 0, 
Eprlte] — Ente] = 0, 
Varr[t] — Varol[t,] = 0, 
and 
Varpl[te] — Varo[te] = 201(1 + pi)(a2 + yoT) — 2po(1 + po)(ai + yiT). 


If we associated smaller variance with better performance, then the 
order (1, 2) is better than the order (2, 1) if 


pi(l + pi) = p2(1 + pe) 


: 7 
oatnT ast yT oe 


This suggests that if other things are equal the queue with smaller 
variability should be served first if we are concerned about the over- 
runs. 

We next evaluate the mean sojourn time, 5;, in queue i, i = 1, 2, 
assuming that it gets served completely in the scheduled slot. We use 
eqs. (14), (16), (17), and (28) and some straightforward but tedious 
algebra to obtain 


B= a +h +5 (Ltn), (38) 
and 
T re i 
Sp = ay + by + 5 (1 + ps) + ai a. (1. 4553): (39) 


Comparing eqs. (38) and (39) with eq. (7), we see that the mean value, 
5, of S; is not affected by the gating mechanism. However, S. does 
depend on the gating. If we use the suffixes 1 and 2 to denote gatings 
G1 and G2, respectively, then 


T 
(1 + p2) — a — piT. (40) 


Under the assumption that the probability of overrun is very small, 
the above difference is negative. Thus the mean sojourn time under 
gating G2 is smaller than under G1. 


3.4 Numerical results 


In this section we use the analysis of Sections 3.1 through 3.3 to 
compute the probability of overrun and the sojourn time distributions 
for some special cases of the simple clocked schedule described earlier. 
For these numerical results we used the algorithm in Ref. 3 for 
inverting Laplace transforms. When the underlying function is not 
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Fig. 7—P,{OVERRUN} as a function of load. 


smooth enough, it may be more appropriate to discretize the problem 
and then use one of the routines for inverting the generating functions. 

Suppose m = 2 and the overhead is zero. The service times are 
deterministic, 5 ms for queue 1 and 1 ms for queue 2. The length of a 
time slot is T = 20 ms. Finally, \; = 2.5 and A, = 12.5). Figure 7 
shows the probability of overrun as a function of \ for gating Gj, gating 
G» (exact), and gating G_ (approximate). For gating G2 the exact and 
approximate results agree very well. For the selected values of the 
parameters the results for gating G1 and G2 differ significantly. The 
probability of overrun using gating G2 is higher than that using Gl. 
This can be explained by the higher variance, as shown in Section 3.3. 
Alternatively, consider the work in queue 1 and queue 2 to be done in 
a time slot (see Fig. 8). For queue 1 the number of jobs to be served is 
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(Ny, Np) 


t 


Fig. 8—Work content in queue 2. 


the number that arrived in time T for both gating mechanisms. The 
number of jobs to be served in queue 2 is the number that arrived in 
time T' for gating G1 and in time T — tj + t,, where tj and ¢, are, 
respectively, times to complete all work in queue 1 in the preceding 
and current intervals. The mean value of T — tj + t, is T, but because 
of the randomness in T — tj + t,, the amount of work arriving at queue 
2 in that time interval has a larger tail than that of the work arriving 
in time T. This explains the larger probability of overrun. This also 
indicates that the difference in the two gating mechanisms is related 
to the variability in tj and ¢,, which corresponds to the variability in 
the work arriving at queue 1. 

Next we reverse the order of service within a time slot. For gating 
G1 the probability of an overrun is not affected by this. However, for 
gating G2 this does change the probability of an overrun. Figure 9 
shows the probability of an overrun as a function of ) for gating G1, 
gating G2 with order (1, 2), and gating G2 with order (2, 1). We 
see considerable reduction in the probability of an overrun with order 
(2, 1). This is expected because queue 2 has less variable work load 
than queue 1 has. Also, as expected, the difference between gating G1 
and G2 is much smaller with order (2, 1) than with order (1, 2). 

We further investigate the effects of the order of service within a 
time slot by considering an example with three queues (m = 3). The 
service times and overheads are deterministic. The parameters are as 
shown in Table IV. Queues 2 and 3 have the same parameters, while 
queue 1 has more variable work load than either queue 2 or 3. The 
probability of an overrun with gatings G1 and G2 and orders (1, 2, 3), 
(2, 1, 3), and (2, 3, 1) are shown in Table IV. Once again, we observe 
that the probability of an overrun decreases as we push the more 
variable queue down in the priority order. 
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Fig. 9—P{OVERRUN} as a function of load. 


Thus, from the moments considerations and from numerical results 
we have the following guidelines when the probability of an overrun 
is of concern: 

1. The difference between the two gating mechanisms is greater 
when the queues with more variable work load are served earlier in a 
time slot. 

2. The probability of an overrun is smaller if queues with more 
variable work load are pushed down in the priority order. 

Of course, in most clocked schedules the differences in the time 
criticality of various tasks determine the order of service. The above 
guidelines, however, are useful when deciding order of service among 
tasks of comparable time criticality. 

We next evaluate the sojourn time distributions numerically. The 
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Table 1V—Probability of overrun 


Number of Queues = 3,T = 20.0 Parameters 


Arrival Service 
Queue Rate Time Overhead 

1 1.0 0.2 1.0 

2 2.0 0.1 1.0 

3 2.0 0.1 1.0 

P{OVERRUN} 
Gating G2 

Approxima- 


tion (First 
Schedule Gating G2 Gating Gl Order) 


(1, 2, 3) 0.027 0.015 0.025 
(2, 1, 3) 0.023 0.015 0.022 
(2, 3, 1) 0.021 0.015 0.020 


case considered here is m = 2, no overhead, and deterministic service 
times. The parameters are \2 = 0.05, bs = 2.0, A, = 0.05, and b, = 1.0. 
In Fig. 10 we have complementary sojourn time distribution, 1 — S2(t), 
for queue 2 under gatings G1 and G2. As indicated earlier the delay 
for queue 2 with gating G1 is larger than with gating G2. 


IV. GENERALIZATIONS AND COMPLEX SCHEDULES 


This section contains various generalizations of the basic method- 
ology described in Section III. Some of these generalizations are aimed 
at relaxing underlying assumptions, while the others look at more 
complex schedules. To avoid presenting a lot of tedious and repetitive 
algebra we will only outline the derivations in this section. The details 
are similar to those in Section III. 


4.1 The effects of queues feeding downward 


In Section III we assumed that the arrivals to different queues form 
independent Poisson processes. In many applications an arrival (job) 
may complete one task and immediately request another. Thus some 
of the task queues may feed the others. In typical situations the high- 
priority queues feed downward within an interval. That is, when a job 
is served in a high-priority queue, it may create an arrival directly to 
the internal queue of another task still to be served in that interval. 
Here, we indicate how to extend the results of Section III to include 
such feed-downs. 

The basic clocked schedule is still the same as in Section III, but 
the arrival processes are now more general. Queue 1 still has an 
exogenous Poisson arrival process at rate \;, 1 = 1 --- m. However, 
when a job is served in queue i, it creates an arrival to the internal 
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Fig. 10—Delay distribution in queue 2. 


queue j with probability p;;. We assume 
Di = 0 J s l, (41) 
and 


»y Di S 1. (42) 


The Poisson arrivals may represent either genuinely exogenous ar- 
rivals or those generated by service completions in other queues in 
previous intervals. The Poisson approximation seems reasonable when 
the number of intervals between the completion of a task and the 
generation of a new one is large. 

We consider the overrun strategy S1 and gating G2 (gating G1 is 
easier to analyze) and obtain the joint transform G(s,, So, --- Sm) of 
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f32sct This can be done recursively as follows: We first obtain G 
in terms of P where 


P(2i,.°** 5 2) = Elzhizh? -.2 2e]. 


We then express P in terms of G with the last argument being zero. 
Since G(0, 0, --- 0) = 1, we get both G and P by iterating on these 
two relations. We implicitly make the assumptions of the type made 
in Section IJ]. Our expressions below are approximations that work 
when the probability that queue 7 overruns is very small, i = 1, ---, 
m-—1. 
For z = (21, 22, ++, 2m), let (2) = —A,(1 — 2), 0 = 1, +++ m. For 

S$ = (81, S2, +++ Sm), let 

Aim(s) = 1; l= 1, vse Mm, 

Bn(s) = Sm, 


Cr(s) = Am(1 -_ Fn (Bm(8))Amm(S)); 
and, fori<j <m-—1, 
Aj j(s) = Aij+i(s)[1 — Dijsr + PijsiFis1(Bjsi(s))], 
Bs) = 8; + Cjrils) + Bj+i(s), 


and 
C,(s) = d,{1 — F(B\(s)) Ajjar(s)]- 
Then 
G(s) = Tl Hu(Bi(s)) 
-P(F,(Bi(s))Ai,(s), aaa: } Fin(Bm(S))Amm(8)) (43) 
and 
B(z) = eG (do(z), «++ 5 bm (2) 0). (44) 


We next study the effect of these feed-downs numerically. We 
consider the case m = 2, no overhead, and deterministic service times 
with b; = by = 1.0. Let pio = p. We consider six cases with p varying 
from 0 to 1 in steps of 0.2. The exogenous arrival rate \2 in queue 2 is 
adjusted so that 


ne = dg + pry = A, 
that is, 
Ae = Ai (1 — p) 
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Fig. 11—P{OVERRUN} with feed-down. 


for all six cases. \; is varied from 0.1 to 0.15. The case p = 0 corresponds 
to no feed-down. Increasing p corresponds to increasing feed-down 
and hence increasing correlation between the queues in an interval. 
The probability of overrun for each of these six cases is presented in 
Fig. 11. At large p, the effect of feed-down is clearly very significant. 


4.2 Complex schedules 


In Section III we analyzed a simple clocked schedule in which each 
high-priority queue is scheduled in each interval. Although such sched- 
ules occur in practice, in most applications a subset of the set of high- 
priority tasks is scheduled in each interval. This subset changes from 
one time interval to the next. The entire schedule may repeat after 
Nperioa time slots. Such a schedule is a clocked schedule with time slot 
length T and schedule period Nperioa. Figure 12 illustrates a schedule 
with period 2. 

In theory we can extend the results of Section III to analyze a 
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Fig. 12—A clocked schedule with period 2T. 


schedule with period Nperioa. For overrun strategy S1 and gating mech- 
anism G1, the analysis is straightforward and can be carried out 
individually for each time slot within a period. For gating G2 with 
overrun strategy S1, note that 


{(NaNgerioat1.ts NrNperioat 1,23 SS oy NnNperioa+1? min & 1)} 


is a time-homogeneous Markov chain and its balance equations can 
be written down as in Section III. We can also use the techniques of 
Section III to obtain transforms G for each time slot in a period in 
terms of P, the joint generating function of the steady-state numbers 
in m queues at the beginning of a period. 

However, for large Nperica both the analysis and numerical evalua- 
tions may be difficult because of the dimensionality. Moreover, we 
need individual analysis for each schedule. When such difficulties 
arise, the approximation discussed below may be an appropriate alter- 
native. This approximation applies uniformly to a wide variety of 
complex schedules and, for each slot, requires the same computational 
effort as for the simple schedule of Section III. 

In order to understand this approximation consider an interval in 
which m, of the m high-priority queues are scheduled. For simplicity 
we number them from 1 through m, the order in which they are 
scheduled for service. Let T denote, as before, the length of the interval 
and, for i = 1, 2 --- m,, let T; denote the nominal time between service 
initiations for queue 1 (that is, if all queues complete service at their 
expected times, then the time between two successive starts of service 
to queue i is T;). For the simple schedule of Section III, 7; = T fori = 
1, --- m. For the schedule in Fig. 12, T, = T, Tz = T3 = 2T. Let 

i-1 


Ti = T; — (aj + 9;T)). (45) 
jal 
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Then the approximation for P is given by 
- Faria - 2) 


P(2, 20, --+ Zm,) =e * (46) 


The relations expressing G(s1, So, «+> Sm,) and Si(s) in terms of P 
remain the same as in Section III. 

Recall that in Section III we calculated the probability of overrun 
for m = 3. The parameter values and the probability of overrun are 
given in Table IV. For the same parameter values we calculated the 
probability of overrun for gating G2 using the approximation described 
above. The results given in the last column indicate that the approx- 
imation works well in this case. A little reflection suggests that for 
schedules with longer periods, the approximation should be even 
better. 


4.3 Other overrun strategies 


In Section III we analyzed clocked schedules with overrun strategy 
S1. In theory, it is possible to extend these results to the schedules 
using overrun strategies S2 and S3. For strategy S2, {(.Nni, Nno, --> 
Nnm):n = 1} is still a time-homogeneous Markov chain. Its balance 
equations can be written down easily, but they do not have the 
triangular structure of those for strategy S1. Thus, these equations 
cannot be solved recursively. They can, however, be solved as a system 
of linear equations after suitably discretizing the underlying distribu- 
tions. For strategy S3 an overrun in one time slot will reduce the time 
available in the following slot. In this case {Nni, Nno, --- 
Nnm, Un):n = 1} is a Markov chain, where N,,; is the number in the 
external queue i when the server visits queue 1 for the nth time and 
U, is the time remaining in that interval. Again, the steady-state 
distribution of this Markov chain can be obtained by solving a system 
of linear equations obtained by suitably discretizing the underlying 
distributions. 

The numerical complexity involved in the above procedures may 
preclude their use for all but small values of m. For large m, some 
approximation or relatively easy to obtain bounds are needed. Note 
that for strategies S2 and S3 the probability of an overrun is bounded 
below by the probability of an overrun for strategy S1, because strategy 
S1 ignores the work carried over from an overrun interval into the 
next. If the amount of overrun is probabilistically small this bound 
must be quite close. 

For strategy S3 we can also get an approximation if we assume that 
overrun does not extend beyond one clock interval. This approxima- 
tion is obtained by adding a dummy queue, say queue 0, at the 
beginning of each schedule interval. The overhead in this queue 
represents the work carried over from the preceding interval. The 
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Table V—P{OVERRUN} for strategy S3 
with T = 20.0, = 0.1, b, = 1.0, and 


b> = 5.0 
P{OVERRUN} 
Approximation 
Approximation Using Dummy 
de Si Queue 
0.025 0.003278 0.003427 
0.05 0.023526 0.026999 


arrival rate to this queue is assumed to be zero. We then use the 
following iterative procedure: 

Step 1—Assume that the overhead in queue 0 is 0. 

Step 2—Using the results of Section III for strategy S1, obtain the 
distribution of the carried over work, (t,. — T)*. 

Step 3—Assume that the overhead in queue 0 has the distribution 
of (tm — T)* obtained in Step 2. 

Repeat Steps 2 and 3 until two successive iterations give “close” values 
for the distribution of (t,, — T)*. 

Numerical values of P{OVERRUN} calculated using strategy S1 
and the approximation for strategy S3 are given in Table V. The 
parameter values are m = 2, no overhead, deterministic service times, 
b, = 1.0, bs = 5.0, A= 0.1, and dhe = 0.025 and 0.05. 


V. CONCLUSIONS 


We presented a methodology to analyze the performance measures 
for the high-priority tasks in clocked schedules. The analysis was exact 
for a simple clocked schedule. For more complex clocked schedules 
this analysis formed the basis for easy to use approximations. 
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